2024-09-15T23:26:08.633 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T23:26:08.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T23:26:08.740 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/snap_schedule_snapdir} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906331' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 client snapdir: .customsnapfuse debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 8921c8ef947807b717db041d85f0761b7914d5fc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: snapdirname: .customsnapkernel syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPW63yeeCV3PrRnSB2Ta0/QdWivC3BBwRQV0Zy0jjI4dGZOQkCJbk7wp+j8HxcZrjoG9TK3AXE8SokgjG22S3Gs= smithi018.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIu5jAMl7C1IRquPwNLCWPcUGzmNAJEImUExGBJm/V4OuS3kjSJTuz8d2QJ1uXcgp6ycZwBVYiHB2oTHu/mGOnY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T23:26:08.740 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T23:26:08.741 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T23:26:08.741 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T23:26:08.743 INFO:teuthology.task.internal:Checking packages... 2024-09-15T23:26:08.764 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T23:26:08.764 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T23:26:08.764 INFO:teuthology.packaging:ref: None 2024-09-15T23:26:08.764 INFO:teuthology.packaging:tag: None 2024-09-15T23:26:08.764 INFO:teuthology.packaging:branch: squid 2024-09-15T23:26:08.764 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:26:08.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T23:26:08.992 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-15T23:26:08.993 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T23:26:08.999 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T23:26:08.999 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T23:26:09.042 INFO:teuthology.task.internal:Saving configuration 2024-09-15T23:26:09.060 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T23:26:09.067 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T23:26:09.092 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:21:24.295504', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPW63yeeCV3PrRnSB2Ta0/QdWivC3BBwRQV0Zy0jjI4dGZOQkCJbk7wp+j8HxcZrjoG9TK3AXE8SokgjG22S3Gs='} 2024-09-15T23:26:09.117 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:21:24.297513', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIu5jAMl7C1IRquPwNLCWPcUGzmNAJEImUExGBJm/V4OuS3kjSJTuz8d2QJ1uXcgp6ycZwBVYiHB2oTHu/mGOnY='} 2024-09-15T23:26:09.117 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T23:26:09.122 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-15T23:26:09.122 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-15T23:26:09.122 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T23:26:09.206 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbd87577be0>, signals=[15]) 2024-09-15T23:26:09.206 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T23:26:09.211 INFO:teuthology.task.internal:Opening connections... 2024-09-15T23:26:09.212 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:26:09.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:26:09.288 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:26:09.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:26:09.364 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T23:26:09.371 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-09-15T23:26:09.389 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-09-15T23:26:09.390 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-09-15T23:26:09.448 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-09-15T23:26:09.448 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-09-15T23:26:09.448 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-09-15T23:26:09.448 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:26:09.448 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-09-15T23:26:09.449 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:26:09.449 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:26:09.449 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-09-15T23:26:09.449 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:26:09.449 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:26:09.449 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:26:09.449 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:26:09.449 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:26:09.449 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:26:09.450 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-09-15T23:26:09.468 DEBUG:teuthology.orchestra.run.smithi018:> uname -m 2024-09-15T23:26:09.486 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2024-09-15T23:26:09.486 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2024-09-15T23:26:09.544 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream" 2024-09-15T23:26:09.544 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="9" 2024-09-15T23:26:09.544 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="9" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:26:09.545 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:26:09.546 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2024-09-15T23:26:09.577 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T23:26:09.589 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T23:26:09.620 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T23:26:09.621 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:26:09.623 DEBUG:teuthology.orchestra.run.smithi018:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:26:09.640 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T23:26:09.649 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T23:26:09.649 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:26:09.681 DEBUG:teuthology.orchestra.run.smithi018:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:26:09.704 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T23:26:09.764 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T23:26:09.783 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-15T23:26:09.783 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-09-15T23:26:09.783 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-09-15T23:26:09.783 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:26:09.784 DEBUG:teuthology.orchestra.run.smithi018:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:26:09.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:26:09.801 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-15T23:26:09.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:26:09.802 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-09-15T23:26:09.858 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-503.el9.x86_64 2024-09-15T23:26:09.858 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-503.el9.x86_64 2024-09-15T23:26:09.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-09-15T23:26:09.860 INFO:teuthology.orchestra.run.smithi018.stdout:5.14.0-503.el9.x86_64 2024-09-15T23:26:09.860 INFO:teuthology.task.kernel:Running kernel on smithi018: 5.14.0-503.el9.x86_64 2024-09-15T23:26:09.860 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum install -y kernel 2024-09-15T23:26:12.400 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-15T23:26:13.516 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-15T23:26:15.467 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-15T23:26:17.950 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-15T23:26:20.051 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 9.8 MB/s | 6.5 MB 00:00 2024-09-15T23:26:22.377 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-15T23:26:22.651 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.5 MB 00:00 2024-09-15T23:26:23.298 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-15T23:26:24.102 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-15T23:26:25.181 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-15T23:26:29.061 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-15T23:26:30.861 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-15T23:26:31.151 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T23:26:31.151 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T23:26:31.247 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-15T23:26:31.251 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-15T23:26:31.251 INFO:teuthology.orchestra.run.smithi018.stdout: Package Architecture Version Repository Size 2024-09-15T23:26:31.251 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-15T23:26:31.251 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-15T23:26:31.251 INFO:teuthology.orchestra.run.smithi018.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T23:26:31.251 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-15T23:26:31.251 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T23:26:31.251 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T23:26:31.251 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T23:26:31.252 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:26:31.252 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-15T23:26:31.252 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-15T23:26:31.252 INFO:teuthology.orchestra.run.smithi018.stdout:Install 4 Packages 2024-09-15T23:26:31.252 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:26:31.252 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 79 M 2024-09-15T23:26:31.253 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 126 M 2024-09-15T23:26:31.253 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-15T23:26:31.379 INFO:teuthology.orchestra.run.smithi018.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 213 kB/s | 23 kB 00:00 2024-09-15T23:26:32.279 INFO:teuthology.orchestra.run.smithi018.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-09-15T23:26:32.830 INFO:teuthology.orchestra.run.smithi018.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-15T23:26:32.960 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T23:26:32.960 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T23:26:33.060 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-15T23:26:33.074 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-15T23:26:33.074 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-15T23:26:33.074 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-15T23:26:33.074 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-15T23:26:33.074 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T23:26:33.074 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-15T23:26:33.075 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T23:26:33.075 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T23:26:33.075 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T23:26:33.075 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:26:33.075 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-15T23:26:33.075 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-15T23:26:33.075 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-15T23:26:33.075 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:26:33.076 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 79 M 2024-09-15T23:26:33.076 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 126 M 2024-09-15T23:26:33.076 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-15T23:26:33.197 INFO:teuthology.orchestra.run.smithi018.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 18 MB/s | 35 MB 00:01 2024-09-15T23:26:33.197 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:26:33.197 INFO:teuthology.orchestra.run.smithi018.stdout:Total 40 MB/s | 79 MB 00:01 2024-09-15T23:26:33.255 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 285 kB/s | 23 kB 00:00 2024-09-15T23:26:33.646 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-15T23:26:33.925 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-15T23:26:34.332 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-15T23:26:34.332 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-15T23:26:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-15T23:26:34.722 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-15T23:26:34.722 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:26:34.722 INFO:teuthology.orchestra.run.smithi007.stdout:Total 48 MB/s | 79 MB 00:01 2024-09-15T23:26:35.330 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-15T23:26:35.331 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-15T23:26:35.429 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-15T23:26:35.817 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-15T23:26:35.817 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-15T23:26:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-15T23:26:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-15T23:26:37.916 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-15T23:26:38.591 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:26:38.672 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:26:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-15T23:26:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:26:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:26:41.479 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:26:41.617 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:26:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:26:43.058 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:26:46.822 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:26:46.868 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:26:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:26:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:26:52.001 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:26:53.376 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:27:31.260 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:27:31.301 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:27:31.600 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:27:31.600 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:27:31.600 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:27:31.600 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:27:32.451 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:27:32.451 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:27:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-15T23:27:32.451 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T23:27:32.452 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T23:27:32.452 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T23:27:32.452 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T23:27:32.452 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:27:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-15T23:27:32.798 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:27:32.882 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:27:32.896 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:27:32.896 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:27:32.897 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T23:27:32.897 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T23:27:32.897 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T23:27:32.897 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-09-15T23:27:32.897 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi018.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T23:27:32.897 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:27:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:27:32.955 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:27:32.956 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:27:32.956 DEBUG:teuthology.orchestra.run.smithi018:> sudo rpm -qi grub2-tools 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Name : grub2-tools 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Epoch : 1 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Version : 2.06 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Release : 82.el9 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Architecture: x86_64 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Group : Unspecified 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Size : 8274670 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:License : GPLv3+ 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T23:27:32.991 INFO:teuthology.orchestra.run.smithi018.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:Packager : builder@centos.org 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:Vendor : CentOS 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:Summary : Support tools for GRUB. 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:Description : 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:hardware devices. 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:27:32.992 INFO:teuthology.orchestra.run.smithi018.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T23:27:32.993 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T23:27:32.993 INFO:teuthology.task.kernel:Updating grub on smithi018 to boot 5.14.0-505.el9.x86_64 2024-09-15T23:27:32.993 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T23:27:33.286 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:27:33.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:27:33.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:27:33.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:27:33.725 INFO:teuthology.orchestra.run.smithi018.stderr:Generating grub configuration file ... 2024-09-15T23:27:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:27:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:27:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-15T23:27:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T23:27:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T23:27:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T23:27:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T23:27:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:27:34.147 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-15T23:27:34.513 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:27:34.601 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:27:34.601 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:27:34.601 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T23:27:34.602 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T23:27:34.602 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T23:27:34.602 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-15T23:27:34.602 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T23:27:34.602 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:27:34.660 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:27:34.660 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:27:34.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-09-15T23:27:34.691 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 82.el9 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8274670 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T23:27:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:27:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T23:27:34.694 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T23:27:34.694 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-505.el9.x86_64 2024-09-15T23:27:34.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T23:27:35.027 INFO:teuthology.orchestra.run.smithi018.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T23:27:35.055 INFO:teuthology.orchestra.run.smithi018.stderr:done 2024-09-15T23:27:35.057 DEBUG:teuthology.orchestra.run.smithi018:> mktemp 2024-09-15T23:27:35.071 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.m2qf4QI3e3 2024-09-15T23:27:35.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.m2qf4QI3e3 2024-09-15T23:27:35.137 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.m2qf4QI3e3 2024-09-15T23:27:35.276 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.m2qf4QI3e3 is 6KB 2024-09-15T23:27:35.323 DEBUG:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.m2qf4QI3e3 2024-09-15T23:27:35.338 DEBUG:teuthology.orchestra.run.smithi018:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T23:27:35.405 INFO:teuthology.orchestra.run.smithi018.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T23:27:35.405 INFO:teuthology.orchestra.run.smithi018.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T23:27:35.405 INFO:teuthology.orchestra.run.smithi018.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T23:27:35.405 INFO:teuthology.orchestra.run.smithi018.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T23:27:35.406 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-09-15T23:27:35.407 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T23:27:35.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo shutdown -r now 2024-09-15T23:27:36.766 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T23:27:36.785 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-09-15T23:27:36.787 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-15T23:27:36.802 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.eYgbHbhsNQ 2024-09-15T23:27:36.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eYgbHbhsNQ 2024-09-15T23:27:36.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.eYgbHbhsNQ 2024-09-15T23:27:37.005 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.eYgbHbhsNQ is 6KB 2024-09-15T23:27:37.052 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.eYgbHbhsNQ 2024-09-15T23:27:37.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T23:27:37.129 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T23:27:37.129 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T23:27:37.129 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T23:27:37.129 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T23:27:37.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T23:27:37.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-09-15T23:28:05.570 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-15T23:28:05.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-15T23:28:05.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:28:07.305 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-15T23:28:07.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-15T23:28:07.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:28:24.125 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.18 2024-09-15T23:28:25.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-15T23:28:33.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-15T23:28:33.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:28:34.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-15T23:28:34.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:29:33.195 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T23:29:34.667 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T23:29:45.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-15T23:29:45.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:29:45.551 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-09-15T23:29:45.884 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-15T23:29:45.884 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-505.el9.x86_64"... 2024-09-15T23:29:45.885 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-09-15T23:29:45.901 INFO:teuthology.orchestra.run.smithi018.stdout:5.14.0-505.el9.x86_64 2024-09-15T23:29:45.901 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T23:29:45.901 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T23:29:45.901 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T23:29:46.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-15T23:29:46.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:29:46.902 DEBUG:teuthology.parallel:result is None 2024-09-15T23:29:47.008 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-15T23:29:47.372 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-15T23:29:47.373 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-15T23:29:47.373 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-15T23:29:47.390 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-505.el9.x86_64 2024-09-15T23:29:47.390 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T23:29:47.390 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T23:29:47.390 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T23:29:48.391 DEBUG:teuthology.parallel:result is None 2024-09-15T23:29:48.391 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T23:29:49.690 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T23:29:49.690 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T23:29:49.693 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T23:29:49.712 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T23:29:50.051 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T23:29:50.512 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T23:29:50.512 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T23:29:50.515 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T23:29:50.692 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T23:29:50.803 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T23:29:50.803 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T23:29:50.806 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T23:29:50.898 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:29:50.898 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:29:50.913 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:29:50.921 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:29:50.922 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T23:29:50.929 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T23:29:50.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T23:29:50.932 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T23:29:50.962 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T23:29:50.971 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T23:29:50.971 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T23:29:51.002 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T23:29:51.021 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:29:51.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:29:51.155 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:29:51.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:29:51.286 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:29:51.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T23:29:51.355 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:29:51.402 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:29:51.456 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:29:51.539 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:29:51.589 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:29:51.589 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T23:29:51.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-09-15T23:29:51.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo service rsyslog restart 2024-09-15T23:29:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:29:51.756 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:29:52.200 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T23:29:52.208 INFO:teuthology.task.internal:Starting timer... 2024-09-15T23:29:52.208 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T23:29:52.249 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T23:29:52.277 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-15T23:29:52.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-09-15T23:29:52.347 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-09-15T23:29:52.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo service auditd rotate 2024-09-15T23:29:52.395 INFO:teuthology.orchestra.run.smithi018.stdout:Rotating logs: 2024-09-15T23:29:52.397 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T23:29:52.397 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-09-15T23:29:52.430 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-09-15T23:29:52.430 DEBUG:teuthology.orchestra.run.smithi018:> /usr/sbin/getenforce 2024-09-15T23:29:52.481 INFO:teuthology.orchestra.run.smithi018.stdout:Permissive 2024-09-15T23:29:52.481 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi018.front.sepia.ceph.com': 'permissive'} 2024-09-15T23:29:52.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:29:52.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:29:52.511 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:29:52.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:29:52.552 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T23:29:52.552 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-09-15T23:29:52.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo /usr/sbin/setenforce permissive 2024-09-15T23:29:52.647 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T23:29:52.656 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T23:29:52.660 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T23:29:52.777 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T23:29:52.788 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T23:29:52.788 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi018.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T23:33:43.766 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi018.front.sepia.ceph.com')] 2024-09-15T23:33:43.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-15T23:33:43.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:33:43.848 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-15T23:33:43.938 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-15T23:33:43.939 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-15T23:33:43.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:33:44.023 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-09-15T23:33:44.097 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-15T23:33:44.098 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T23:33:44.113 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T23:33:44.113 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T23:33:44.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:33:44.117 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T23:33:44.117 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:33:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T23:33:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T23:33:44.170 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T23:33:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T23:33:44.193 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-09-15T23:33:44.205 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-09-15T23:33:44.212 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: ntpd: command not found 2024-09-15T23:33:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T23:33:44.224 INFO:teuthology.orchestra.run.smithi018.stdout:506 Cannot talk to daemon 2024-09-15T23:33:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T23:33:44.239 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T23:33:44.254 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T23:33:44.294 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:33:44.298 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:33:44.298 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-15T23:33:44.298 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:33:44.298 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:33:44.298 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:33:44.298 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:33:44.317 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:33:44.322 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:33:44.322 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-09-15T23:33:44.322 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:33:44.322 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:33:44.322 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:33:44.322 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:33:44.323 INFO:teuthology.run_tasks:Running task install... 2024-09-15T23:33:44.331 DEBUG:teuthology.task.install:project ceph 2024-09-15T23:33:44.331 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-15T23:33:44.332 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T23:33:44.332 INFO:teuthology.task.install:Using flavor: default 2024-09-15T23:33:44.338 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T23:33:44.338 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-15T23:33:44.339 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T23:33:44.339 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:33:44.341 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T23:33:44.341 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:33:44.483 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T23:33:44.483 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T23:33:44.490 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T23:33:44.490 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T23:33:44.592 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T23:33:44.592 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:33:44.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T23:33:44.601 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T23:33:44.602 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:33:44.602 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T23:33:44.623 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T23:33:44.623 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T23:33:44.630 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T23:33:44.630 DEBUG:teuthology.orchestra.run.smithi018:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T23:33:44.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T23:33:44.697 DEBUG:teuthology.orchestra.run.smithi018:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T23:33:44.769 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T23:33:44.775 DEBUG:teuthology.orchestra.run.smithi018:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T23:33:44.835 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-09-15T23:33:44.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-09-15T23:33:44.840 INFO:teuthology.orchestra.run.smithi018.stdout:check_obsoletes = 1 2024-09-15T23:33:44.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum clean all 2024-09-15T23:33:45.076 INFO:teuthology.orchestra.run.smithi018.stdout:45 files removed 2024-09-15T23:33:45.076 INFO:teuthology.orchestra.run.smithi007.stdout:45 files removed 2024-09-15T23:33:45.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T23:33:45.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T23:33:45.666 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 319 kB/s | 83 kB 00:00 2024-09-15T23:33:45.686 INFO:teuthology.orchestra.run.smithi018.stdout:ceph packages for x86_64 297 kB/s | 83 kB 00:00 2024-09-15T23:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-15T23:33:45.978 INFO:teuthology.orchestra.run.smithi018.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-15T23:33:46.226 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-15T23:33:46.245 INFO:teuthology.orchestra.run.smithi018.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-15T23:33:46.673 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.2 MB 00:00 2024-09-15T23:33:46.776 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-15T23:33:48.644 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-15T23:33:51.047 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 20 MB 00:03 2024-09-15T23:33:53.111 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-15T23:33:54.637 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 92 kB/s | 19 kB 00:00 2024-09-15T23:33:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-15T23:33:56.039 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 6.5 MB 00:01 2024-09-15T23:33:57.656 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-15T23:33:58.644 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-15T23:34:01.406 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-15T23:34:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:34:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:34:03.830 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T23:34:03.831 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T23:34:03.832 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T23:34:03.833 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T23:34:03.835 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T23:34:03.837 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T23:34:03.838 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T23:34:03.905 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-15T23:34:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-15T23:34:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-15T23:34:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-15T23:34:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-15T23:34:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T23:34:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T23:34:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T23:34:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T23:34:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T23:34:03.928 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T23:34:03.929 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T23:34:03.930 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T23:34:03.931 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T23:34:03.932 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T23:34:03.933 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T23:34:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T23:34:03.935 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T23:34:03.936 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T23:34:03.937 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T23:34:03.938 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T23:34:03.939 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-09-15T23:34:03.940 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T23:34:03.940 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:34:03.940 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-15T23:34:03.940 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-15T23:34:03.940 INFO:teuthology.orchestra.run.smithi007.stdout:Install 148 Packages 2024-09-15T23:34:03.940 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 8 Packages 2024-09-15T23:34:03.940 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:34:03.941 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 226 M 2024-09-15T23:34:03.941 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-15T23:34:04.266 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-15T23:34:04.516 INFO:teuthology.orchestra.run.smithi007.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-15T23:34:04.741 INFO:teuthology.orchestra.run.smithi007.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-15T23:34:04.816 INFO:teuthology.orchestra.run.smithi007.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.9 MB/s | 145 kB 00:00 2024-09-15T23:34:04.891 INFO:teuthology.orchestra.run.smithi007.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-15T23:34:05.025 INFO:teuthology.orchestra.run.smithi007.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-15T23:34:05.067 INFO:teuthology.orchestra.run.smithi007.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-15T23:34:05.434 INFO:teuthology.orchestra.run.smithi007.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-15T23:34:05.784 INFO:teuthology.orchestra.run.smithi007.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 15 MB/s | 21 MB 00:01 2024-09-15T23:34:05.818 INFO:teuthology.orchestra.run.smithi007.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 761 kB/s | 25 kB 00:00 2024-09-15T23:34:05.968 INFO:teuthology.orchestra.run.smithi007.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 20 MB/s | 11 MB 00:00 2024-09-15T23:34:06.152 INFO:teuthology.orchestra.run.smithi007.stdout:(11/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 16 MB/s | 17 MB 00:01 2024-09-15T23:34:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.1 MB/s | 228 kB 00:00 2024-09-15T23:34:06.194 INFO:teuthology.orchestra.run.smithi007.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 649 kB/s | 27 kB 00:00 2024-09-15T23:34:06.219 INFO:teuthology.orchestra.run.smithi007.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 752 kB/s | 32 kB 00:00 2024-09-15T23:34:06.261 INFO:teuthology.orchestra.run.smithi007.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 751 kB 00:00 2024-09-15T23:34:06.287 INFO:teuthology.orchestra.run.smithi007.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.4 MB/s | 163 kB 00:00 2024-09-15T23:34:06.312 INFO:teuthology.orchestra.run.smithi007.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-15T23:34:06.354 INFO:teuthology.orchestra.run.smithi007.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 7.4 MB/s | 506 kB 00:00 2024-09-15T23:34:06.396 INFO:teuthology.orchestra.run.smithi007.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 1.1 MB/s | 45 kB 00:00 2024-09-15T23:34:06.463 INFO:teuthology.orchestra.run.smithi007.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.0 MB/s | 136 kB 00:00 2024-09-15T23:34:06.538 INFO:teuthology.orchestra.run.smithi007.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.1 MB/s | 161 kB 00:00 2024-09-15T23:34:06.622 INFO:teuthology.orchestra.run.smithi007.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-15T23:34:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:(23/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-15T23:34:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 2.9 MB/s | 300 kB 00:00 2024-09-15T23:34:06.737 INFO:teuthology.orchestra.run.smithi018.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:34:06.738 INFO:teuthology.orchestra.run.smithi018.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:34:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-15T23:34:06.752 INFO:teuthology.orchestra.run.smithi018.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T23:34:06.753 INFO:teuthology.orchestra.run.smithi018.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T23:34:06.755 INFO:teuthology.orchestra.run.smithi018.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T23:34:06.756 INFO:teuthology.orchestra.run.smithi018.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T23:34:06.758 INFO:teuthology.orchestra.run.smithi018.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T23:34:06.761 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T23:34:06.762 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T23:34:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-15T23:34:06.823 INFO:teuthology.orchestra.run.smithi007.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-15T23:34:06.830 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-15T23:34:06.860 INFO:teuthology.orchestra.run.smithi018.stdout:====================================================================================== 2024-09-15T23:34:06.860 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-15T23:34:06.860 INFO:teuthology.orchestra.run.smithi018.stdout:====================================================================================== 2024-09-15T23:34:06.860 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T23:34:06.861 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T23:34:06.862 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T23:34:06.862 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T23:34:06.862 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T23:34:06.862 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T23:34:06.862 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T23:34:06.862 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T23:34:06.862 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T23:34:06.862 INFO:teuthology.orchestra.run.smithi018.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T23:34:06.862 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T23:34:06.863 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T23:34:06.864 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T23:34:06.865 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T23:34:06.866 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T23:34:06.867 INFO:teuthology.orchestra.run.smithi018.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T23:34:06.867 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T23:34:06.867 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T23:34:06.867 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T23:34:06.867 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T23:34:06.867 INFO:teuthology.orchestra.run.smithi018.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T23:34:06.867 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T23:34:06.867 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T23:34:06.868 INFO:teuthology.orchestra.run.smithi018.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T23:34:06.869 INFO:teuthology.orchestra.run.smithi018.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T23:34:06.869 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T23:34:06.869 INFO:teuthology.orchestra.run.smithi018.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T23:34:06.869 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T23:34:06.869 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T23:34:06.870 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T23:34:06.871 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T23:34:06.872 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T23:34:06.873 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T23:34:06.874 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T23:34:06.874 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T23:34:06.874 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T23:34:06.874 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T23:34:06.874 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T23:34:06.874 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T23:34:06.874 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T23:34:06.874 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T23:34:06.875 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:Installing weak dependencies: 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-15T23:34:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:====================================================================================== 2024-09-15T23:34:06.877 INFO:teuthology.orchestra.run.smithi018.stdout:Install 148 Packages 2024-09-15T23:34:06.877 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 8 Packages 2024-09-15T23:34:06.877 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:34:06.877 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 226 M 2024-09-15T23:34:06.877 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-15T23:34:06.899 INFO:teuthology.orchestra.run.smithi007.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 425 kB/s | 32 kB 00:00 2024-09-15T23:34:06.957 INFO:teuthology.orchestra.run.smithi007.stdout:(29/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-15T23:34:06.982 INFO:teuthology.orchestra.run.smithi007.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.7 MB/s | 146 kB 00:00 2024-09-15T23:34:07.266 INFO:teuthology.orchestra.run.smithi007.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 12 MB/s | 3.6 MB 00:00 2024-09-15T23:34:07.367 INFO:teuthology.orchestra.run.smithi007.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 19 MB/s | 7.4 MB 00:00 2024-09-15T23:34:07.400 INFO:teuthology.orchestra.run.smithi007.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 1.8 MB/s | 249 kB 00:00 2024-09-15T23:34:07.425 INFO:teuthology.orchestra.run.smithi007.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 845 kB/s | 49 kB 00:00 2024-09-15T23:34:07.451 INFO:teuthology.orchestra.run.smithi007.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 327 kB/s | 16 kB 00:00 2024-09-15T23:34:07.476 INFO:teuthology.orchestra.run.smithi007.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.5 MB/s | 285 kB 00:00 2024-09-15T23:34:07.484 INFO:teuthology.orchestra.run.smithi018.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 57 kB/s | 6.5 kB 00:00 2024-09-15T23:34:07.510 INFO:teuthology.orchestra.run.smithi007.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 13 MB/s | 765 kB 00:00 2024-09-15T23:34:07.535 INFO:teuthology.orchestra.run.smithi007.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 311 kB/s | 18 kB 00:00 2024-09-15T23:34:07.677 INFO:teuthology.orchestra.run.smithi007.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-15T23:34:07.794 INFO:teuthology.orchestra.run.smithi007.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-15T23:34:07.826 INFO:teuthology.orchestra.run.smithi018.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 2.6 MB/s | 896 kB 00:00 2024-09-15T23:34:07.828 INFO:teuthology.orchestra.run.smithi007.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-15T23:34:07.876 INFO:teuthology.orchestra.run.smithi018.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.8 MB/s | 145 kB 00:00 2024-09-15T23:34:08.002 INFO:teuthology.orchestra.run.smithi018.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 8.5 MB/s | 5.4 MB 00:00 2024-09-15T23:34:08.110 INFO:teuthology.orchestra.run.smithi018.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 1.5 MB 00:00 2024-09-15T23:34:08.287 INFO:teuthology.orchestra.run.smithi007.stdout:(42/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 20 MB/s | 49 MB 00:02 2024-09-15T23:34:08.294 INFO:teuthology.orchestra.run.smithi018.stdout:(6/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 5.7 MB/s | 2.4 MB 00:00 2024-09-15T23:34:08.312 INFO:teuthology.orchestra.run.smithi007.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 489 kB/s | 253 kB 00:00 2024-09-15T23:34:08.346 INFO:teuthology.orchestra.run.smithi007.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-09-15T23:34:08.394 INFO:teuthology.orchestra.run.smithi018.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-15T23:34:08.396 INFO:teuthology.orchestra.run.smithi007.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-15T23:34:08.430 INFO:teuthology.orchestra.run.smithi007.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-15T23:34:08.455 INFO:teuthology.orchestra.run.smithi007.stdout:(47/156): python3-ply-3.11-14.el9.noarch.rpm 635 kB/s | 106 kB 00:00 2024-09-15T23:34:08.480 INFO:teuthology.orchestra.run.smithi007.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-15T23:34:08.581 INFO:teuthology.orchestra.run.smithi007.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-15T23:34:08.631 INFO:teuthology.orchestra.run.smithi007.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-15T23:34:08.656 INFO:teuthology.orchestra.run.smithi007.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-15T23:34:08.681 INFO:teuthology.orchestra.run.smithi007.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-15T23:34:08.707 INFO:teuthology.orchestra.run.smithi007.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-15T23:34:08.732 INFO:teuthology.orchestra.run.smithi007.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-15T23:34:08.757 INFO:teuthology.orchestra.run.smithi007.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-15T23:34:08.782 INFO:teuthology.orchestra.run.smithi007.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-15T23:34:08.808 INFO:teuthology.orchestra.run.smithi007.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-15T23:34:08.833 INFO:teuthology.orchestra.run.smithi007.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-09-15T23:34:08.892 INFO:teuthology.orchestra.run.smithi007.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-15T23:34:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-15T23:34:08.951 INFO:teuthology.orchestra.run.smithi007.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-15T23:34:08.976 INFO:teuthology.orchestra.run.smithi007.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-15T23:34:08.994 INFO:teuthology.orchestra.run.smithi018.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 21 MB 00:01 2024-09-15T23:34:09.051 INFO:teuthology.orchestra.run.smithi007.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-09-15T23:34:09.077 INFO:teuthology.orchestra.run.smithi007.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-15T23:34:09.102 INFO:teuthology.orchestra.run.smithi007.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-15T23:34:09.111 INFO:teuthology.orchestra.run.smithi018.stdout:(9/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9. 15 MB/s | 11 MB 00:00 2024-09-15T23:34:09.128 INFO:teuthology.orchestra.run.smithi018.stdout:(10/156): ceph-selinux-19.1.1-326.g8921c8ef.el9 189 kB/s | 25 kB 00:00 2024-09-15T23:34:09.178 INFO:teuthology.orchestra.run.smithi018.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.5 MB/s | 228 kB 00:00 2024-09-15T23:34:09.237 INFO:teuthology.orchestra.run.smithi018.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 464 kB/s | 27 kB 00:00 2024-09-15T23:34:09.287 INFO:teuthology.orchestra.run.smithi018.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 636 kB/s | 32 kB 00:00 2024-09-15T23:34:09.380 INFO:teuthology.orchestra.run.smithi018.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 8.0 MB/s | 751 kB 00:00 2024-09-15T23:34:09.455 INFO:teuthology.orchestra.run.smithi018.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.1 MB/s | 163 kB 00:00 2024-09-15T23:34:09.519 INFO:teuthology.orchestra.run.smithi007.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2024-09-15T23:34:09.530 INFO:teuthology.orchestra.run.smithi018.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.7 MB/s | 127 kB 00:00 2024-09-15T23:34:09.730 INFO:teuthology.orchestra.run.smithi018.stdout:(17/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 17 MB 00:01 2024-09-15T23:34:09.764 INFO:teuthology.orchestra.run.smithi018.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 2.1 MB/s | 506 kB 00:00 2024-09-15T23:34:09.822 INFO:teuthology.orchestra.run.smithi018.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 776 kB/s | 45 kB 00:00 2024-09-15T23:34:09.906 INFO:teuthology.orchestra.run.smithi018.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-15T23:34:09.953 INFO:teuthology.orchestra.run.smithi007.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-15T23:34:10.031 INFO:teuthology.orchestra.run.smithi018.stdout:(21/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-15T23:34:10.056 INFO:teuthology.orchestra.run.smithi018.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.0 MB/s | 161 kB 00:00 2024-09-15T23:34:10.082 INFO:teuthology.orchestra.run.smithi018.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 6.2 MB/s | 320 kB 00:00 2024-09-15T23:34:10.107 INFO:teuthology.orchestra.run.smithi018.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.9 MB/s | 300 kB 00:00 2024-09-15T23:34:10.132 INFO:teuthology.orchestra.run.smithi018.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-15T23:34:10.181 INFO:teuthology.orchestra.run.smithi018.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-15T23:34:10.219 INFO:teuthology.orchestra.run.smithi007.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.3 MB/s | 910 kB 00:00 2024-09-15T23:34:10.242 INFO:teuthology.orchestra.run.smithi018.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.8 MB/s | 170 kB 00:00 2024-09-15T23:34:10.295 INFO:teuthology.orchestra.run.smithi007.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-15T23:34:10.300 INFO:teuthology.orchestra.run.smithi018.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 19 MB/s | 3.1 MB 00:00 2024-09-15T23:34:10.326 INFO:teuthology.orchestra.run.smithi018.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 381 kB/s | 32 kB 00:00 2024-09-15T23:34:10.351 INFO:teuthology.orchestra.run.smithi018.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-15T23:34:10.387 INFO:teuthology.orchestra.run.smithi007.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-09-15T23:34:10.520 INFO:teuthology.orchestra.run.smithi007.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.4 MB/s | 4.9 MB 00:01 2024-09-15T23:34:10.579 INFO:teuthology.orchestra.run.smithi007.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-15T23:34:10.626 INFO:teuthology.orchestra.run.smithi018.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 12 MB/s | 3.6 MB 00:00 2024-09-15T23:34:10.727 INFO:teuthology.orchestra.run.smithi018.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.4 MB/s | 249 kB 00:00 2024-09-15T23:34:10.737 INFO:teuthology.orchestra.run.smithi007.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-15T23:34:10.780 INFO:teuthology.orchestra.run.smithi007.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-15T23:34:10.835 INFO:teuthology.orchestra.run.smithi018.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-15T23:34:10.852 INFO:teuthology.orchestra.run.smithi018.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 395 kB/s | 49 kB 00:00 2024-09-15T23:34:10.863 INFO:teuthology.orchestra.run.smithi007.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-15T23:34:10.869 INFO:teuthology.orchestra.run.smithi018.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 494 kB/s | 16 kB 00:00 2024-09-15T23:34:10.903 INFO:teuthology.orchestra.run.smithi018.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.5 MB/s | 285 kB 00:00 2024-09-15T23:34:10.945 INFO:teuthology.orchestra.run.smithi018.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 9.9 MB/s | 765 kB 00:00 2024-09-15T23:34:10.947 INFO:teuthology.orchestra.run.smithi007.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-15T23:34:10.962 INFO:teuthology.orchestra.run.smithi018.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 311 kB/s | 18 kB 00:00 2024-09-15T23:34:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 618 kB/s | 26 kB 00:00 2024-09-15T23:34:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-09-15T23:34:11.106 INFO:teuthology.orchestra.run.smithi007.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-15T23:34:11.148 INFO:teuthology.orchestra.run.smithi007.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 838 kB/s | 35 kB 00:00 2024-09-15T23:34:11.429 INFO:teuthology.orchestra.run.smithi018.stdout:(39/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 21 MB/s | 49 MB 00:02 2024-09-15T23:34:11.454 INFO:teuthology.orchestra.run.smithi018.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 157 kB/s | 80 kB 00:00 2024-09-15T23:34:11.546 INFO:teuthology.orchestra.run.smithi018.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-15T23:34:11.571 INFO:teuthology.orchestra.run.smithi018.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-15T23:34:11.597 INFO:teuthology.orchestra.run.smithi018.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-15T23:34:11.622 INFO:teuthology.orchestra.run.smithi018.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-15T23:34:11.647 INFO:teuthology.orchestra.run.smithi018.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-15T23:34:11.672 INFO:teuthology.orchestra.run.smithi018.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-15T23:34:11.698 INFO:teuthology.orchestra.run.smithi018.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-15T23:34:11.740 INFO:teuthology.orchestra.run.smithi018.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 6.3 MB/s | 1.2 MB 00:00 2024-09-15T23:34:11.840 INFO:teuthology.orchestra.run.smithi018.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-15T23:34:11.865 INFO:teuthology.orchestra.run.smithi018.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-15T23:34:11.891 INFO:teuthology.orchestra.run.smithi018.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-15T23:34:11.924 INFO:teuthology.orchestra.run.smithi018.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-15T23:34:11.958 INFO:teuthology.orchestra.run.smithi018.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 487 kB/s | 33 kB 00:00 2024-09-15T23:34:12.007 INFO:teuthology.orchestra.run.smithi007.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.4 MB/s | 5.8 MB 00:01 2024-09-15T23:34:12.008 INFO:teuthology.orchestra.run.smithi018.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-15T23:34:12.033 INFO:teuthology.orchestra.run.smithi018.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-09-15T23:34:12.060 INFO:teuthology.orchestra.run.smithi018.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-15T23:34:12.099 INFO:teuthology.orchestra.run.smithi007.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-15T23:34:12.100 INFO:teuthology.orchestra.run.smithi018.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-15T23:34:12.126 INFO:teuthology.orchestra.run.smithi018.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-15T23:34:12.182 INFO:teuthology.orchestra.run.smithi007.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-15T23:34:12.210 INFO:teuthology.orchestra.run.smithi018.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-15T23:34:12.241 INFO:teuthology.orchestra.run.smithi007.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-15T23:34:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.3 MB/s | 581 kB 00:00 2024-09-15T23:34:12.277 INFO:teuthology.orchestra.run.smithi018.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 365 kB/s | 24 kB 00:00 2024-09-15T23:34:12.310 INFO:teuthology.orchestra.run.smithi018.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-15T23:34:12.333 INFO:teuthology.orchestra.run.smithi007.stdout:(85/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-09-15T23:34:12.336 INFO:teuthology.orchestra.run.smithi018.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-15T23:34:12.358 INFO:teuthology.orchestra.run.smithi007.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-09-15T23:34:12.361 INFO:teuthology.orchestra.run.smithi018.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-15T23:34:12.378 INFO:teuthology.orchestra.run.smithi018.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 924 kB/s | 38 kB 00:00 2024-09-15T23:34:12.400 INFO:teuthology.orchestra.run.smithi007.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-09-15T23:34:12.459 INFO:teuthology.orchestra.run.smithi007.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-09-15T23:34:12.501 INFO:teuthology.orchestra.run.smithi007.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-15T23:34:12.543 INFO:teuthology.orchestra.run.smithi007.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-09-15T23:34:12.568 INFO:teuthology.orchestra.run.smithi007.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 803 kB/s | 54 kB 00:00 2024-09-15T23:34:12.610 INFO:teuthology.orchestra.run.smithi007.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-15T23:34:12.652 INFO:teuthology.orchestra.run.smithi007.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-15T23:34:12.703 INFO:teuthology.orchestra.run.smithi018.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-09-15T23:34:12.805 INFO:teuthology.orchestra.run.smithi007.stdout:(94/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-15T23:34:12.836 INFO:teuthology.orchestra.run.smithi007.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-09-15T23:34:12.861 INFO:teuthology.orchestra.run.smithi007.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 632 kB/s | 36 kB 00:00 2024-09-15T23:34:12.887 INFO:teuthology.orchestra.run.smithi007.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-15T23:34:12.928 INFO:teuthology.orchestra.run.smithi007.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-15T23:34:12.987 INFO:teuthology.orchestra.run.smithi007.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-15T23:34:13.012 INFO:teuthology.orchestra.run.smithi007.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-15T23:34:13.054 INFO:teuthology.orchestra.run.smithi007.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 952 kB/s | 64 kB 00:00 2024-09-15T23:34:13.080 INFO:teuthology.orchestra.run.smithi007.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-09-15T23:34:13.129 INFO:teuthology.orchestra.run.smithi018.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-15T23:34:13.155 INFO:teuthology.orchestra.run.smithi007.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-15T23:34:13.214 INFO:teuthology.orchestra.run.smithi007.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-15T23:34:13.256 INFO:teuthology.orchestra.run.smithi007.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-15T23:34:13.298 INFO:teuthology.orchestra.run.smithi007.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-09-15T23:34:13.323 INFO:teuthology.orchestra.run.smithi007.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-15T23:34:13.357 INFO:teuthology.orchestra.run.smithi007.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-15T23:34:13.382 INFO:teuthology.orchestra.run.smithi007.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-15T23:34:13.404 INFO:teuthology.orchestra.run.smithi018.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.2 MB/s | 910 kB 00:00 2024-09-15T23:34:13.407 INFO:teuthology.orchestra.run.smithi007.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T23:34:13.432 INFO:teuthology.orchestra.run.smithi007.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-15T23:34:13.473 INFO:teuthology.orchestra.run.smithi018.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2024-09-15T23:34:13.474 INFO:teuthology.orchestra.run.smithi007.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-15T23:34:13.513 INFO:teuthology.orchestra.run.smithi018.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-15T23:34:13.524 INFO:teuthology.orchestra.run.smithi007.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-15T23:34:13.550 INFO:teuthology.orchestra.run.smithi007.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T23:34:13.575 INFO:teuthology.orchestra.run.smithi007.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-15T23:34:13.600 INFO:teuthology.orchestra.run.smithi007.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-15T23:34:13.613 INFO:teuthology.orchestra.run.smithi018.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.9 MB/s | 4.9 MB 00:01 2024-09-15T23:34:13.634 INFO:teuthology.orchestra.run.smithi007.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-15T23:34:13.676 INFO:teuthology.orchestra.run.smithi007.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 8.4 MB/s | 358 kB 00:00 2024-09-15T23:34:13.688 INFO:teuthology.orchestra.run.smithi018.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-15T23:34:13.710 INFO:teuthology.orchestra.run.smithi007.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-15T23:34:13.743 INFO:teuthology.orchestra.run.smithi007.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.4 MB/s | 220 kB 00:00 2024-09-15T23:34:13.819 INFO:teuthology.orchestra.run.smithi007.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-09-15T23:34:13.853 INFO:teuthology.orchestra.run.smithi007.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-15T23:34:13.872 INFO:teuthology.orchestra.run.smithi018.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-15T23:34:13.878 INFO:teuthology.orchestra.run.smithi007.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 434 kB/s | 11 kB 00:00 2024-09-15T23:34:13.903 INFO:teuthology.orchestra.run.smithi007.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-15T23:34:13.905 INFO:teuthology.orchestra.run.smithi018.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-15T23:34:13.920 INFO:teuthology.orchestra.run.smithi007.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-15T23:34:13.937 INFO:teuthology.orchestra.run.smithi007.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-15T23:34:13.954 INFO:teuthology.orchestra.run.smithi007.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-15T23:34:14.013 INFO:teuthology.orchestra.run.smithi007.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-09-15T23:34:14.014 INFO:teuthology.orchestra.run.smithi018.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-15T23:34:14.038 INFO:teuthology.orchestra.run.smithi007.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T23:34:14.064 INFO:teuthology.orchestra.run.smithi007.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-15T23:34:14.089 INFO:teuthology.orchestra.run.smithi007.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-15T23:34:14.098 INFO:teuthology.orchestra.run.smithi018.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-15T23:34:14.122 INFO:teuthology.orchestra.run.smithi007.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-15T23:34:14.131 INFO:teuthology.orchestra.run.smithi018.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 768 kB/s | 26 kB 00:00 2024-09-15T23:34:14.148 INFO:teuthology.orchestra.run.smithi007.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-15T23:34:14.165 INFO:teuthology.orchestra.run.smithi018.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 579 kB/s | 19 kB 00:00 2024-09-15T23:34:14.173 INFO:teuthology.orchestra.run.smithi007.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-15T23:34:14.198 INFO:teuthology.orchestra.run.smithi007.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-15T23:34:14.232 INFO:teuthology.orchestra.run.smithi007.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-15T23:34:14.240 INFO:teuthology.orchestra.run.smithi018.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-15T23:34:14.257 INFO:teuthology.orchestra.run.smithi007.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 524 kB/s | 31 kB 00:00 2024-09-15T23:34:14.274 INFO:teuthology.orchestra.run.smithi018.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-09-15T23:34:14.282 INFO:teuthology.orchestra.run.smithi007.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-15T23:34:14.308 INFO:teuthology.orchestra.run.smithi007.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-15T23:34:14.333 INFO:teuthology.orchestra.run.smithi007.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-15T23:34:14.359 INFO:teuthology.orchestra.run.smithi007.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-15T23:34:14.384 INFO:teuthology.orchestra.run.smithi007.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-15T23:34:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-15T23:34:14.434 INFO:teuthology.orchestra.run.smithi007.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 804 kB/s | 20 kB 00:00 2024-09-15T23:34:14.460 INFO:teuthology.orchestra.run.smithi007.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-15T23:34:14.485 INFO:teuthology.orchestra.run.smithi007.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-15T23:34:14.543 INFO:teuthology.orchestra.run.smithi007.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-15T23:34:14.786 INFO:teuthology.orchestra.run.smithi007.stdout:(148/156): librados2-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-15T23:34:14.853 INFO:teuthology.orchestra.run.smithi007.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-15T23:34:15.178 INFO:teuthology.orchestra.run.smithi007.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 5.9 MB/s | 2.3 MB 00:00 2024-09-15T23:34:15.199 INFO:teuthology.orchestra.run.smithi018.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-09-15T23:34:15.237 INFO:teuthology.orchestra.run.smithi007.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 4.7 MB/s | 1.8 MB 00:00 2024-09-15T23:34:15.262 INFO:teuthology.orchestra.run.smithi007.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 488 kB/s | 41 kB 00:00 2024-09-15T23:34:15.308 INFO:teuthology.orchestra.run.smithi018.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-15T23:34:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-15T23:34:15.416 INFO:teuthology.orchestra.run.smithi018.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.5 MB/s | 6.0 MB 00:01 2024-09-15T23:34:15.438 INFO:teuthology.orchestra.run.smithi007.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 7.7 MB/s | 1.4 MB 00:00 2024-09-15T23:34:15.458 INFO:teuthology.orchestra.run.smithi018.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-15T23:34:15.484 INFO:teuthology.orchestra.run.smithi018.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-15T23:34:15.509 INFO:teuthology.orchestra.run.smithi018.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-15T23:34:15.584 INFO:teuthology.orchestra.run.smithi018.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-15T23:34:15.610 INFO:teuthology.orchestra.run.smithi018.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-09-15T23:34:15.629 INFO:teuthology.orchestra.run.smithi007.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.8 MB/s | 2.4 MB 00:00 2024-09-15T23:34:15.651 INFO:teuthology.orchestra.run.smithi018.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-15T23:34:15.693 INFO:teuthology.orchestra.run.smithi018.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-15T23:34:15.735 INFO:teuthology.orchestra.run.smithi018.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-15T23:34:15.777 INFO:teuthology.orchestra.run.smithi018.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-15T23:34:15.894 INFO:teuthology.orchestra.run.smithi018.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-15T23:34:15.920 INFO:teuthology.orchestra.run.smithi018.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 311 kB/s | 44 kB 00:00 2024-09-15T23:34:15.962 INFO:teuthology.orchestra.run.smithi018.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 862 kB/s | 36 kB 00:00 2024-09-15T23:34:16.012 INFO:teuthology.orchestra.run.smithi018.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-15T23:34:16.029 INFO:teuthology.orchestra.run.smithi018.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-09-15T23:34:16.071 INFO:teuthology.orchestra.run.smithi018.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-15T23:34:16.129 INFO:teuthology.orchestra.run.smithi018.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-15T23:34:16.155 INFO:teuthology.orchestra.run.smithi018.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-15T23:34:16.188 INFO:teuthology.orchestra.run.smithi018.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-15T23:34:16.205 INFO:teuthology.orchestra.run.smithi018.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-15T23:34:16.306 INFO:teuthology.orchestra.run.smithi018.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-15T23:34:16.381 INFO:teuthology.orchestra.run.smithi018.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-15T23:34:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-15T23:34:16.431 INFO:teuthology.orchestra.run.smithi018.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-15T23:34:16.473 INFO:teuthology.orchestra.run.smithi018.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-15T23:34:16.524 INFO:teuthology.orchestra.run.smithi018.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-15T23:34:16.549 INFO:teuthology.orchestra.run.smithi018.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-15T23:34:16.575 INFO:teuthology.orchestra.run.smithi018.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T23:34:16.608 INFO:teuthology.orchestra.run.smithi018.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-15T23:34:16.675 INFO:teuthology.orchestra.run.smithi018.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-15T23:34:16.725 INFO:teuthology.orchestra.run.smithi018.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-15T23:34:16.742 INFO:teuthology.orchestra.run.smithi018.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.5 MB/s | 43 kB 00:00 2024-09-15T23:34:16.784 INFO:teuthology.orchestra.run.smithi018.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-09-15T23:34:16.801 INFO:teuthology.orchestra.run.smithi018.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-09-15T23:34:16.818 INFO:teuthology.orchestra.run.smithi018.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 421 kB/s | 14 kB 00:00 2024-09-15T23:34:16.843 INFO:teuthology.orchestra.run.smithi018.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-15T23:34:16.869 INFO:teuthology.orchestra.run.smithi018.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-15T23:34:16.903 INFO:teuthology.orchestra.run.smithi018.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-15T23:34:16.936 INFO:teuthology.orchestra.run.smithi018.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-15T23:34:16.964 INFO:teuthology.orchestra.run.smithi018.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-15T23:34:16.987 INFO:teuthology.orchestra.run.smithi018.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-15T23:34:17.012 INFO:teuthology.orchestra.run.smithi018.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 704 kB/s | 18 kB 00:00 2024-09-15T23:34:17.038 INFO:teuthology.orchestra.run.smithi018.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 924 kB/s | 23 kB 00:00 2024-09-15T23:34:17.063 INFO:teuthology.orchestra.run.smithi018.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-15T23:34:17.088 INFO:teuthology.orchestra.run.smithi018.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 785 kB/s | 20 kB 00:00 2024-09-15T23:34:17.147 INFO:teuthology.orchestra.run.smithi018.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-15T23:34:17.172 INFO:teuthology.orchestra.run.smithi018.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T23:34:17.206 INFO:teuthology.orchestra.run.smithi018.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-15T23:34:17.239 INFO:teuthology.orchestra.run.smithi018.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-15T23:34:17.265 INFO:teuthology.orchestra.run.smithi018.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-09-15T23:34:17.298 INFO:teuthology.orchestra.run.smithi018.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-09-15T23:34:17.315 INFO:teuthology.orchestra.run.smithi018.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-15T23:34:17.340 INFO:teuthology.orchestra.run.smithi018.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-15T23:34:17.358 INFO:teuthology.orchestra.run.smithi018.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 668 kB/s | 28 kB 00:00 2024-09-15T23:34:17.375 INFO:teuthology.orchestra.run.smithi018.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 909 kB/s | 31 kB 00:00 2024-09-15T23:34:17.400 INFO:teuthology.orchestra.run.smithi018.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-09-15T23:34:17.417 INFO:teuthology.orchestra.run.smithi018.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-15T23:34:17.434 INFO:teuthology.orchestra.run.smithi018.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-15T23:34:17.459 INFO:teuthology.orchestra.run.smithi018.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-15T23:34:17.493 INFO:teuthology.orchestra.run.smithi018.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-09-15T23:34:17.518 INFO:teuthology.orchestra.run.smithi018.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-15T23:34:17.543 INFO:teuthology.orchestra.run.smithi018.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-15T23:34:17.577 INFO:teuthology.orchestra.run.smithi018.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-15T23:34:17.652 INFO:teuthology.orchestra.run.smithi018.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-15T23:34:17.713 INFO:teuthology.orchestra.run.smithi007.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.2 MB/s | 16 MB 00:05 2024-09-15T23:34:17.718 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:34:17.718 INFO:teuthology.orchestra.run.smithi007.stdout:Total 16 MB/s | 226 MB 00:13 2024-09-15T23:34:17.719 INFO:teuthology.orchestra.run.smithi018.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-15T23:34:17.895 INFO:teuthology.orchestra.run.smithi018.stdout:(148/156): librados2-19.1.1-326.g8921c8ef.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-15T23:34:18.011 INFO:teuthology.orchestra.run.smithi018.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-15T23:34:18.079 INFO:teuthology.orchestra.run.smithi018.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-15T23:34:18.121 INFO:teuthology.orchestra.run.smithi018.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 980 kB/s | 41 kB 00:00 2024-09-15T23:34:18.171 INFO:teuthology.orchestra.run.smithi018.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-15T23:34:18.229 INFO:teuthology.orchestra.run.smithi018.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-15T23:34:18.288 INFO:teuthology.orchestra.run.smithi018.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 6.5 MB/s | 1.8 MB 00:00 2024-09-15T23:34:18.338 INFO:teuthology.orchestra.run.smithi018.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-15T23:34:18.992 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-15T23:34:19.162 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-15T23:34:19.162 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-15T23:34:20.972 INFO:teuthology.orchestra.run.smithi018.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.1 MB/s | 16 MB 00:05 2024-09-15T23:34:20.977 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:34:20.977 INFO:teuthology.orchestra.run.smithi018.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-15T23:34:21.613 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-15T23:34:21.613 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-15T23:34:22.247 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-15T23:34:22.418 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-15T23:34:22.418 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-15T23:34:24.766 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-15T23:34:24.766 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-15T23:34:28.181 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-15T23:34:28.295 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T23:34:28.431 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T23:34:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T23:34:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T23:34:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T23:34:28.783 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T23:34:29.139 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T23:34:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:34:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:34:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:34:29.545 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:34:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T23:34:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T23:34:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:34:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:34:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:34:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:34:30.022 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:34:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:34:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:34:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:34:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T23:34:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:34:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:34:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:34:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T23:34:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T23:34:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T23:34:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T23:34:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T23:34:31.355 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-15T23:34:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T23:34:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T23:34:31.469 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T23:34:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T23:34:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T23:34:31.513 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T23:34:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T23:34:31.580 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T23:34:31.604 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T23:34:31.620 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T23:34:31.653 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T23:34:31.656 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T23:34:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T23:34:31.714 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T23:34:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T23:34:31.817 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T23:34:31.857 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T23:34:31.901 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T23:34:31.929 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T23:34:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T23:34:32.053 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T23:34:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T23:34:32.130 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T23:34:32.179 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:34:32.275 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T23:34:32.305 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:34:32.330 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:34:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T23:34:32.423 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:34:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:34:32.653 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T23:34:32.716 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T23:34:32.743 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:34:32.849 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:34:32.877 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:34:32.896 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T23:34:32.941 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:34:32.970 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:34:32.991 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:34:33.003 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T23:34:33.047 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T23:34:33.098 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T23:34:33.136 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T23:34:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T23:34:33.182 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:34:33.357 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:34:33.361 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T23:34:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T23:34:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T23:34:33.485 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T23:34:33.545 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T23:34:33.590 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T23:34:33.611 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:34:33.702 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T23:34:33.703 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:34:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T23:34:33.936 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T23:34:33.948 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:34:33.981 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T23:34:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T23:34:34.064 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T23:34:34.158 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T23:34:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T23:34:34.193 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T23:34:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T23:34:34.236 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T23:34:34.264 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T23:34:34.312 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T23:34:34.371 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T23:34:34.379 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T23:34:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T23:34:34.419 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T23:34:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T23:34:34.462 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T23:34:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T23:34:34.495 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T23:34:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T23:34:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T23:34:34.594 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T23:34:34.603 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T23:34:34.654 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T23:34:34.684 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T23:34:34.728 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T23:34:34.765 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T23:34:34.799 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T23:34:34.815 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T23:34:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T23:34:34.855 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T23:34:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T23:34:34.899 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T23:34:34.917 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T23:34:34.926 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T23:34:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T23:34:34.987 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T23:34:35.000 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T23:34:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T23:34:35.059 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T23:34:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T23:34:35.114 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T23:34:35.136 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T23:34:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T23:34:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T23:34:35.282 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T23:34:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T23:34:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T23:34:35.337 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T23:34:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T23:34:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T23:34:35.902 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T23:34:35.984 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T23:34:36.019 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T23:34:36.071 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T23:34:36.117 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T23:34:36.159 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T23:34:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T23:34:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T23:34:36.312 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T23:34:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T23:34:36.370 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T23:34:36.426 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T23:34:36.437 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T23:34:36.490 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T23:34:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T23:34:36.538 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T23:34:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T23:34:36.574 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T23:34:36.630 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T23:34:36.647 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T23:34:36.665 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T23:34:36.703 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T23:34:36.709 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T23:34:36.751 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T23:34:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T23:34:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T23:34:36.996 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T23:34:37.002 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T23:34:37.086 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T23:34:37.127 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T23:34:37.234 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T23:34:37.278 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T23:34:37.318 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T23:34:37.359 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T23:34:37.384 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T23:34:37.489 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T23:34:37.628 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T23:34:37.670 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T23:34:37.695 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T23:34:37.714 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T23:34:37.754 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T23:34:37.797 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T23:34:37.807 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T23:34:37.853 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T23:34:37.883 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T23:34:37.918 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T23:34:37.951 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T23:34:37.993 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T23:34:38.027 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T23:34:38.117 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T23:34:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T23:34:38.205 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T23:34:38.312 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T23:34:38.374 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T23:34:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T23:34:39.136 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T23:34:39.187 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T23:34:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T23:34:39.220 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T23:34:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T23:34:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T23:34:39.325 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T23:34:39.377 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T23:34:39.421 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T23:34:39.516 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T23:34:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T23:34:39.606 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T23:34:39.651 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:34:39.741 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T23:34:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:34:39.781 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T23:34:39.781 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T23:34:39.820 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T23:34:39.862 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T23:34:40.326 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T23:34:40.362 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:34:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:34:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T23:34:40.556 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T23:34:40.675 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T23:34:42.000 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T23:34:42.077 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T23:34:42.105 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T23:34:42.138 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T23:34:42.465 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T23:34:42.509 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:34:42.603 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:34:42.633 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T23:34:42.669 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T23:34:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:34:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:34:43.178 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T23:34:43.214 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:34:43.269 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:34:43.299 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T23:34:43.572 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:34:43.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:34:43.707 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:34:43.708 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T23:34:43.708 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T23:34:43.708 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:34:43.749 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:34:45.702 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:34:45.737 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:34:46.319 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:34:46.357 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:34:46.450 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:34:46.450 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T23:34:46.450 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T23:34:46.450 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:34:46.483 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:34:56.125 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:34:56.125 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-09-15T23:34:56.126 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-09-15T23:34:56.126 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-09-15T23:34:56.126 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-09-15T23:34:56.126 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-09-15T23:34:56.126 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-09-15T23:34:56.126 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-09-15T23:34:56.126 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:34:56.310 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:34:56.328 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:34:56.328 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:34:56.328 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T23:34:56.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:34:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:34:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:34:57.892 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:34:57.892 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T23:34:57.892 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:34:57.893 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:34:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:34:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:34:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:34:58.158 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:34:58.158 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T23:34:58.158 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:34:58.158 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:34:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:34:58.616 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:34:58.641 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:34:58.641 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:34:58.642 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T23:34:58.642 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:34:58.642 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:34:58.642 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:34:58.679 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:34:58.679 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /sys 2024-09-15T23:34:58.679 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /proc 2024-09-15T23:34:58.679 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /mnt 2024-09-15T23:34:58.679 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /var/tmp 2024-09-15T23:34:58.679 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /home 2024-09-15T23:34:58.679 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /root 2024-09-15T23:34:58.679 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /tmp 2024-09-15T23:34:58.679 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:34:58.747 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T23:34:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T23:34:58.877 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T23:34:58.962 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:34:58.980 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:34:58.980 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:34:58.981 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T23:34:58.981 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:34:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T23:34:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T23:34:59.207 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T23:34:59.304 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T23:34:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T23:34:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T23:34:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T23:34:59.577 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T23:35:00.025 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:35:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:35:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T23:35:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T23:35:00.174 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:35:00.515 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:35:00.543 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:35:00.544 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:00.544 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T23:35:00.544 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:35:00.544 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:35:00.544 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:00.782 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:35:00.807 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:35:00.807 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:00.807 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T23:35:00.808 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:35:00.808 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:35:00.808 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:01.266 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:35:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:35:01.291 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:35:01.291 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:01.291 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T23:35:01.292 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:35:01.292 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:35:01.292 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:01.318 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:35:01.405 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T23:35:01.463 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T23:35:01.527 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T23:35:01.731 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T23:35:01.776 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T23:35:01.857 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T23:35:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T23:35:02.105 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T23:35:02.162 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T23:35:02.199 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T23:35:02.227 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T23:35:02.336 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:35:02.360 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:35:02.469 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:35:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T23:35:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:35:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:35:02.696 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:02.696 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T23:35:02.696 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:35:02.696 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:35:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:02.710 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:35:02.743 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:35:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:35:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:35:02.775 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T23:35:02.841 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T23:35:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T23:35:02.865 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:35:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T23:35:03.894 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T23:35:03.924 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:35:03.949 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:35:03.949 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:03.949 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T23:35:03.949 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:35:03.949 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:35:03.950 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:03.972 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:35:04.000 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:35:04.009 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:35:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:35:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T23:35:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:35:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:35:04.100 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:04.101 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T23:35:04.101 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:35:04.101 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:35:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:04.418 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:35:04.443 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:35:04.443 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:04.443 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T23:35:04.443 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:35:04.443 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:35:04.443 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:35:05.051 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:35:05.160 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:35:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T23:35:05.370 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:35:05.395 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:35:05.395 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:05.395 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T23:35:05.395 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:35:05.396 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:35:05.396 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:05.453 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:35:05.468 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:35:05.576 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T23:35:05.636 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T23:35:06.615 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T23:35:06.648 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:35:06.673 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:35:06.674 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:06.674 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T23:35:06.674 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:35:06.674 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:35:06.674 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:06.715 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:35:06.740 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:35:06.740 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:06.740 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T23:35:06.740 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:06.791 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:35:06.816 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:35:06.816 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:06.816 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T23:35:06.816 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:35:06.816 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:35:06.816 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:07.133 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:35:07.158 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:35:07.158 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:35:07.158 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T23:35:07.159 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:35:07.159 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:35:07.159 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:09.296 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T23:35:09.338 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T23:35:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T23:35:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T23:35:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T23:35:09.624 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T23:35:09.668 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T23:35:09.698 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T23:35:09.728 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T23:35:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T23:35:10.429 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:35:10.472 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:35:10.495 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:35:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T23:35:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T23:35:11.542 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T23:35:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T23:35:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T23:35:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T23:35:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T23:35:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:35:11.825 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:35:11.825 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:35:11.909 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:35:11.934 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T23:35:11.940 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:35:11.940 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:35:11.995 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T23:35:12.045 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T23:35:12.086 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T23:35:12.118 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T23:35:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:35:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:35:12.145 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T23:35:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:35:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:35:12.353 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T23:35:12.392 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T23:35:12.424 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T23:35:12.452 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T23:35:12.506 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T23:35:13.152 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:35:13.195 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:35:13.218 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:35:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T23:35:14.234 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T23:35:14.282 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T23:35:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T23:35:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T23:35:14.437 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T23:35:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T23:35:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:35:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:35:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:35:14.657 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:35:14.707 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T23:35:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:35:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:35:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:35:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:35:14.993 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:35:15.062 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:35:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:35:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T23:35:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:35:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T23:35:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T23:35:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T23:35:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T23:35:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T23:35:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T23:35:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T23:35:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T23:35:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T23:35:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T23:35:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T23:35:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T23:35:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T23:35:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T23:35:18.080 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T23:35:18.081 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T23:35:18.081 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T23:35:20.577 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:35:20.577 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:20.577 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-09-15T23:35:20.577 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T23:35:20.577 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.578 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.579 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T23:35:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T23:35:20.581 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T23:35:20.582 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T23:35:20.583 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T23:35:20.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T23:35:20.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T23:35:20.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T23:35:20.588 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T23:35:20.589 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T23:35:20.590 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T23:35:20.590 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T23:35:20.590 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:20.590 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-15T23:35:20.715 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:35:20.715 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T23:35:20.715 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T23:35:20.715 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T23:35:20.715 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T23:35:20.715 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T23:35:20.715 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T23:35:20.715 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T23:35:20.715 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T23:35:20.716 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T23:35:20.717 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T23:35:20.717 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T23:35:20.717 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T23:35:20.717 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T23:35:20.717 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T23:35:20.717 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T23:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T23:35:20.730 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T23:35:20.731 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T23:35:20.732 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T23:35:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T23:35:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T23:35:20.735 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T23:35:20.736 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T23:35:20.737 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T23:35:20.738 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T23:35:20.738 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T23:35:20.738 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T23:35:20.738 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T23:35:20.738 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T23:35:20.738 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T23:35:20.738 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T23:35:20.738 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T23:35:20.738 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T23:35:20.739 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T23:35:20.740 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T23:35:20.740 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T23:35:21.164 DEBUG:teuthology.parallel:result is None 2024-09-15T23:35:23.388 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:35:23.388 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:23.388 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2024-09-15T23:35:23.388 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T23:35:23.388 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T23:35:23.388 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.389 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.390 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.390 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.390 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.390 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.390 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.390 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.390 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.390 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.391 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.391 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.391 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.391 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.391 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.391 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.391 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T23:35:23.392 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T23:35:23.393 INFO:teuthology.orchestra.run.smithi018.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T23:35:23.394 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T23:35:23.394 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T23:35:23.394 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T23:35:23.394 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.394 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.394 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T23:35:23.394 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.394 INFO:teuthology.orchestra.run.smithi018.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T23:35:23.394 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T23:35:23.395 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.396 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T23:35:23.397 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T23:35:23.398 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T23:35:23.399 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T23:35:23.399 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T23:35:23.399 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T23:35:23.399 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T23:35:23.399 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T23:35:23.400 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T23:35:23.401 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T23:35:23.402 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T23:35:23.403 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T23:35:23.403 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:23.403 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-15T23:35:23.821 DEBUG:teuthology.parallel:result is None 2024-09-15T23:35:23.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:35:23.961 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T23:35:23.989 INFO:teuthology.orchestra.run.smithi007.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T23:35:23.989 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T23:35:23.989 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T23:35:23.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:35:24.129 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T23:35:24.153 INFO:teuthology.orchestra.run.smithi018.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T23:35:24.153 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T23:35:24.153 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T23:35:24.156 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T23:35:24.157 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:24.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T23:35:24.190 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:24.190 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T23:35:24.221 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T23:35:24.221 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:24.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T23:35:24.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T23:35:24.330 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:24.331 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T23:35:24.357 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T23:35:24.424 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T23:35:24.424 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:24.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T23:35:24.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T23:35:24.521 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:24.521 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T23:35:24.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T23:35:24.615 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T23:35:24.616 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:24.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T23:35:24.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T23:35:24.713 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:24.713 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T23:35:24.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T23:35:24.805 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T23:35:24.882 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T23:35:24.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-09-15T23:35:24.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 777 /var/log/ceph 2024-09-15T23:35:24.910 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T23:35:24.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T23:35:24.953 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T23:35:24.979 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T23:35:24.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T23:35:25.022 DEBUG:teuthology.orchestra.run.smithi018:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T23:35:25.053 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-15T23:35:25.053 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-15T23:35:25.054 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-15T23:35:25.054 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:25.054 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-15T23:35:25.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-15T23:35:25.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-15T23:35:25.248 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:25.248 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-15T23:35:25.264 DEBUG:teuthology.orchestra.run.smithi018:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-15T23:35:25.375 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-15T23:35:25.406 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T23:35:25.406 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T23:35:25.407 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/snap_schedule_snapdir}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7906331', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, 'kclient': {'snapdirname': '.customsnapkernel', 'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3393, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'sleep_before_teardown': 0, 'subset': '294/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPW63yeeCV3PrRnSB2Ta0/QdWivC3BBwRQV0Zy0jjI4dGZOQkCJbk7wp+j8HxcZrjoG9TK3AXE8SokgjG22S3Gs=', 'smithi018.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIu5jAMl7C1IRquPwNLCWPcUGzmNAJEImUExGBJm/V4OuS3kjSJTuz8d2QJ1uXcgp6ycZwBVYiHB2oTHu/mGOnY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_21:24:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T23:35:25.407 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T23:35:25.410 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T23:35:25.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T23:35:25.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T23:35:25.523 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:25.523 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T23:35:25.548 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T23:35:25.548 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-09-15T23:35:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T23:35:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:35:25.605 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-15T23:35:25.606 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:35:25.606 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:35:25.606 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-15 23:35:12.635264835 +0000 2024-09-15T23:35:25.606 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-15 23:33:22.834509549 +0000 2024-09-15T23:35:25.606 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-15 23:33:22.834509549 +0000 2024-09-15T23:35:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-15 23:33:22.834509549 +0000 2024-09-15T23:35:25.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T23:35:25.670 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-15T23:35:25.670 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-15T23:35:25.670 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.9243e-05 s, 5.2 MB/s 2024-09-15T23:35:25.671 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T23:35:25.729 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-09-15T23:35:25.785 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T23:35:25.785 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:35:25.785 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T23:35:25.785 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:35:25.785 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:35:25.786 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-15 23:35:12.635264835 +0000 2024-09-15T23:35:25.786 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-15 23:33:23.175504831 +0000 2024-09-15T23:35:25.786 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-15 23:33:23.175504831 +0000 2024-09-15T23:35:25.786 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-15 23:33:23.175504831 +0000 2024-09-15T23:35:25.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T23:35:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-15T23:35:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-15T23:35:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000111211 s, 4.6 MB/s 2024-09-15T23:35:25.850 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T23:35:25.906 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-09-15T23:35:25.962 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T23:35:25.962 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:35:25.962 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-15T23:35:25.962 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:35:25.962 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:35:25.962 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-15 23:35:12.636264821 +0000 2024-09-15T23:35:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-15 23:33:23.492500444 +0000 2024-09-15T23:35:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-15 23:33:23.492500444 +0000 2024-09-15T23:35:25.963 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-15 23:33:23.492500444 +0000 2024-09-15T23:35:25.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T23:35:26.026 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-15T23:35:26.027 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-15T23:35:26.027 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.9446e-05 s, 5.1 MB/s 2024-09-15T23:35:26.028 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T23:35:26.085 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-09-15T23:35:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T23:35:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:35:26.141 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-15T23:35:26.141 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:35:26.141 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:35:26.141 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-15 23:35:12.636264821 +0000 2024-09-15T23:35:26.142 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-15 23:33:23.821495892 +0000 2024-09-15T23:35:26.142 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-15 23:33:23.821495892 +0000 2024-09-15T23:35:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-15 23:33:23.821495892 +0000 2024-09-15T23:35:26.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T23:35:26.205 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-15T23:35:26.205 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-15T23:35:26.205 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000106296 s, 4.8 MB/s 2024-09-15T23:35:26.207 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T23:35:26.263 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:26.263 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:26.263 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T23:35:26.282 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T23:35:26.282 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_1 2024-09-15T23:35:26.339 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T23:35:26.339 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:35:26.339 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T23:35:26.340 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:35:26.340 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:35:26.340 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-15 23:35:15.428593415 +0000 2024-09-15T23:35:26.340 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-15 23:33:22.817802728 +0000 2024-09-15T23:35:26.340 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-15 23:33:22.817802728 +0000 2024-09-15T23:35:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-15 23:33:22.817802728 +0000 2024-09-15T23:35:26.340 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T23:35:26.407 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-15T23:35:26.407 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-15T23:35:26.407 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000115651 s, 4.4 MB/s 2024-09-15T23:35:26.408 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T23:35:26.467 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_2 2024-09-15T23:35:26.524 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T23:35:26.524 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:35:26.524 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-15T23:35:26.524 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:35:26.524 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:35:26.524 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-15 23:35:15.429593404 +0000 2024-09-15T23:35:26.525 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-15 23:33:23.154799201 +0000 2024-09-15T23:35:26.525 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-15 23:33:23.154799201 +0000 2024-09-15T23:35:26.525 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-15 23:33:23.154799201 +0000 2024-09-15T23:35:26.525 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T23:35:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-15T23:35:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-15T23:35:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000109855 s, 4.7 MB/s 2024-09-15T23:35:26.592 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T23:35:26.650 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_3 2024-09-15T23:35:26.707 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T23:35:26.708 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:35:26.708 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-15T23:35:26.708 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:35:26.708 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:35:26.708 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-15 23:35:15.429593404 +0000 2024-09-15T23:35:26.708 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-15 23:33:23.473795863 +0000 2024-09-15T23:35:26.708 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-15 23:33:23.473795863 +0000 2024-09-15T23:35:26.708 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-15 23:33:23.473795863 +0000 2024-09-15T23:35:26.708 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T23:35:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-15T23:35:26.773 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-15T23:35:26.773 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 8.8936e-05 s, 5.8 MB/s 2024-09-15T23:35:26.774 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T23:35:26.832 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_4 2024-09-15T23:35:26.889 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T23:35:26.889 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:35:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-15T23:35:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:35:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:35:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-15 23:35:15.429593404 +0000 2024-09-15T23:35:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-15 23:33:23.803792410 +0000 2024-09-15T23:35:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-15 23:33:23.803792410 +0000 2024-09-15T23:35:26.889 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-15 23:33:23.803792410 +0000 2024-09-15T23:35:26.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T23:35:26.953 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-15T23:35:26.953 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-15T23:35:26.953 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000104941 s, 4.9 MB/s 2024-09-15T23:35:26.954 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T23:35:27.012 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:27.012 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi018.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T23:35:27.012 INFO:tasks.ceph:Generating config... 2024-09-15T23:35:27.013 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-15T23:35:27.013 INFO:tasks.ceph:[client] client snapdir = .customsnapfuse 2024-09-15T23:35:27.013 INFO:tasks.ceph:[client] debug client = 20 2024-09-15T23:35:27.013 INFO:tasks.ceph:[client] debug ms = 1 2024-09-15T23:35:27.013 INFO:tasks.ceph:[client] pid file = 2024-09-15T23:35:27.013 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-15T23:35:27.013 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-15T23:35:27.014 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T23:35:27.014 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T23:35:27.015 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T23:35:27.015 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T23:35:27.015 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T23:35:27.015 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-15T23:35:27.015 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-15T23:35:27.015 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-15T23:35:27.015 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T23:35:27.015 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T23:35:27.015 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T23:35:27.015 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T23:35:27.015 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T23:35:27.015 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T23:35:27.015 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T23:35:27.015 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T23:35:27.016 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T23:35:27.016 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T23:35:27.016 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T23:35:27.016 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T23:35:27.016 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-15T23:35:27.016 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T23:35:27.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T23:35:27.067 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T23:35:27.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T23:35:27.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T23:35:27.140 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.18'), ('mon.c', '[v2:172.21.15.18:3301,v1:172.21.15.18:6790]')] 2024-09-15T23:35:27.141 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.18,[v2:172.21.15.18:3301,v1:172.21.15.18:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-15T23:35:27.142 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:27.142 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T23:35:27.197 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.18 --addv c '[v2:172.21.15.18:3301,v1:172.21.15.18:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 9da25f4c-4673-468c-a40c-f01392dffc9b 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 9da25f4c-4673-468c-a40c-f01392dffc9b 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-09-15T23:35:27.284962+0000 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-15T23:35:27.284962+0000 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 17 (quincy) 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-09-15T23:35:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-15T23:35:27.286 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.b 2024-09-15T23:35:27.286 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] mon.c 2024-09-15T23:35:27.286 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T23:35:27.287 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T23:35:27.302 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9da25f4c-4673-468c-a40c-f01392dffc9b... 2024-09-15T23:35:27.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T23:35:27.345 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T23:35:27.388 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-15T23:35:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-15T23:35:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-15T23:35:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-15T23:35:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.18,[v2:172.21.15.18:3301,v1:172.21.15.18:6790]" 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 9da25f4c-4673-468c-a40c-f01392dffc9b 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-15T23:35:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-15T23:35:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-15T23:35:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-15T23:35:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-15T23:35:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 20 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 20 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-15T23:35:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 10 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-15T23:35:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-15T23:35:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-15T23:35:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: mon down mkfs grace = 300 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on pool no app = False 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-09-15T23:35:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: client snapdir = .customsnapfuse 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-15T23:35:27.400 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-15T23:35:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-15T23:35:27.402 INFO:teuthology.orchestra.run.smithi018.stdout:[global] 2024-09-15T23:35:27.402 INFO:teuthology.orchestra.run.smithi018.stdout: chdir = "" 2024-09-15T23:35:27.402 INFO:teuthology.orchestra.run.smithi018.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T23:35:27.402 INFO:teuthology.orchestra.run.smithi018.stdout: auth supported = cephx 2024-09-15T23:35:27.402 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.402 INFO:teuthology.orchestra.run.smithi018.stdout: filestore xattr use omap = true 2024-09-15T23:35:27.402 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.402 INFO:teuthology.orchestra.run.smithi018.stdout: mon clock drift allowed = 1.000 2024-09-15T23:35:27.402 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: osd crush chooseleaf type = 0 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: auth debug = true 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: ms die on old message = true 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: ms die on bug = true 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: mon pg warn max object skew = 0 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T23:35:27.403 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: osd pool default size = 2 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd allow primary affinity = true 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd allow pg remap = true 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on legacy crush tunables = false 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on crush straw calc version zero = false 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on no sortbitwise = false 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on osd down out interval zero = false 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on too few osds = false 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T23:35:27.404 INFO:teuthology.orchestra.run.smithi018.stdout: mon_allow_pool_size_one = true 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: osd default data pool replay window = 5 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: mon allow pool delete = true 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: mon cluster log file level = debug 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: debug asserts on shutdown = true 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: mon health detail to clog = false 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: mon host = "172.21.15.7,172.21.15.18,[v2:172.21.15.18:3301,v1:172.21.15.18:6790]" 2024-09-15T23:35:27.405 INFO:teuthology.orchestra.run.smithi018.stdout: ms die on skipped message = False 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: fsid = 9da25f4c-4673-468c-a40c-f01392dffc9b 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout:[osd] 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: osd journal size = 100 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: osd scrub load threshold = 5.0 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: osd scrub max interval = 600 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: osd mclock profile = high_recovery_ops 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: osd recover clone overlap = true 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: osd recovery max chunk = 1048576 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug shutdown = true 2024-09-15T23:35:27.406 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug op order = true 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug verify stray on activate = true 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug trim objects = true 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: osd open classes on start = true 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug pg log writeout = true 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: osd deep scrub update digest min age = 30 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: osd map max advance = 10 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: journal zero on create = true 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: filestore ondisk finisher threads = 3 2024-09-15T23:35:27.407 INFO:teuthology.orchestra.run.smithi018.stdout: filestore apply finisher threads = 3 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: bdev debug aio = true 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug misdirected ops = true 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: bluestore block size = 96636764160 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: bluestore fsck on mount = True 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: debug bluefs = 20 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: debug bluestore = 20 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: debug osd = 20 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: debug rocksdb = 10 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd full ratio = 0.9 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd nearfull ratio = 0.8 2024-09-15T23:35:27.408 INFO:teuthology.orchestra.run.smithi018.stdout: osd failsafe full ratio = 0.95 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: osd objectstore = bluestore 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: osd op complaint time = 180 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout:[mgr] 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: debug mgr = 20 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: debug mon = 20 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: debug auth = 20 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min pgs per osd = 4 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min bytes per osd = 10 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: mgr/telemetry/nag = false 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: debug client = 20 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.409 INFO:teuthology.orchestra.run.smithi018.stdout:[mon] 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: debug mon = 20 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: debug paxos = 20 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: debug auth = 20 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: mon data avail warn = 5 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: mon mgr mkfs grace = 240 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min pgs per osd = 4 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd reporter subtree level = osd 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd prime pg temp = true 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min bytes per osd = 10 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T23:35:27.410 INFO:teuthology.orchestra.run.smithi018.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: # 1m isn't quite enough 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: mon_down_mkfs_grace = 2m 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_filestore_osds = false 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: mon down mkfs grace = 300 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: mon op complaint time = 120 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on pool no app = False 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-15T23:35:27.411 INFO:teuthology.orchestra.run.smithi018.stdout:[client] 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: rgw cache enabled = true 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: rgw enable ops log = true 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: rgw enable usage log = true 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: client mount timeout = 600 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: client snapdir = .customsnapfuse 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: debug client = 20 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: pid file = "" 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: rados mon op timeout = 900 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout: rados osd op timeout = 900 2024-09-15T23:35:27.412 INFO:teuthology.orchestra.run.smithi018.stdout:[mon.a] 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout:[mon.b] 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout:[mon.c] 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.a] 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.c] 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.b] 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.d] 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout:[mds] 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout: debug mds = 20 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout: debug mds balancer = 20 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout: mds debug frag = True 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout: mds debug scatterstat = True 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout: mds op complaint time = 180 2024-09-15T23:35:27.413 INFO:teuthology.orchestra.run.smithi018.stdout: mds verify scatter = True 2024-09-15T23:35:27.414 INFO:teuthology.orchestra.run.smithi018.stdout: osd op complaint time = 180 2024-09-15T23:35:27.414 INFO:teuthology.orchestra.run.smithi018.stdout: rados mon op timeout = 900 2024-09-15T23:35:27.414 INFO:teuthology.orchestra.run.smithi018.stdout: rados osd op timeout = 900 2024-09-15T23:35:27.415 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T23:35:27.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T23:35:27.511 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T23:35:27.511 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:27.511 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T23:35:27.526 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:27.527 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T23:35:27.582 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:35:27.582 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:27.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T23:35:27.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T23:35:27.658 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:27.658 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:27.714 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:35:27.714 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:27.714 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T23:35:27.714 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T23:35:27.751 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:27.751 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:27.807 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T23:35:27.808 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T23:35:27.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T23:35:27.860 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T23:35:27.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T23:35:27.920 INFO:teuthology.orchestra.run.smithi018.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T23:35:27.923 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T23:35:27.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-15T23:35:27.976 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-15T23:35:27.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-15T23:35:28.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-15T23:35:28.097 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-15T23:35:28.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-15T23:35:28.125 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-15T23:35:28.178 INFO:teuthology.orchestra.run.smithi018.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-15T23:35:28.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-15T23:35:28.207 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-15T23:35:28.299 INFO:teuthology.orchestra.run.smithi018.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-15T23:35:28.302 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-15T23:35:28.327 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T23:35:28.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T23:35:28.370 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T23:35:28.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-15T23:35:28.466 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-15T23:35:28.479 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-15T23:35:28.521 INFO:teuthology.orchestra.run.smithi018.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-15T23:35:28.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-15T23:35:28.617 INFO:teuthology.orchestra.run.smithi018.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-15T23:35:28.630 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T23:35:28.630 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi018.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T23:35:28.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T23:35:28.655 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:28.655 INFO:tasks.ceph:role: osd.0 2024-09-15T23:35:28.656 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:35:28.656 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T23:35:28.720 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T23:35:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T23:35:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T23:35:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T23:35:28.721 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T23:35:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-15T23:35:28.721 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T23:35:28.721 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T23:35:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T23:35:28.721 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T23:35:28.752 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-15T23:35:28.755 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-15T23:35:28.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T23:35:28.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T23:35:28.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T23:35:28.961 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:28.961 INFO:tasks.ceph:role: osd.1 2024-09-15T23:35:28.961 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:35:28.961 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T23:35:29.028 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T23:35:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T23:35:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T23:35:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T23:35:29.029 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T23:35:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-15T23:35:29.029 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T23:35:29.029 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T23:35:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T23:35:29.029 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T23:35:29.059 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-15T23:35:29.062 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-15T23:35:29.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T23:35:29.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T23:35:29.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T23:35:29.268 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:29.268 INFO:tasks.ceph:role: osd.2 2024-09-15T23:35:29.268 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:35:29.269 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T23:35:29.334 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T23:35:29.366 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-15T23:35:29.368 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-15T23:35:29.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T23:35:29.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T23:35:29.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T23:35:29.574 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:29.574 INFO:tasks.ceph:role: osd.3 2024-09-15T23:35:29.574 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:35:29.574 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T23:35:29.640 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T23:35:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T23:35:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T23:35:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T23:35:29.641 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T23:35:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-15T23:35:29.641 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T23:35:29.641 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T23:35:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T23:35:29.641 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T23:35:29.673 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-15T23:35:29.675 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-15T23:35:29.675 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-15T23:35:29.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T23:35:29.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:29.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:29.907+0000 7f7fcd813640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T23:35:29.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:29.907+0000 7f7fcd813640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T23:35:29.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:29.907+0000 7f7fcd813640 -1 bdev(0x5624b0a87800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-15T23:35:29.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:29.907+0000 7f7fcd813640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T23:35:33.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T23:35:33.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:33.203+0000 7fcf0600d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T23:35:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:33.203+0000 7fcf0600d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T23:35:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:33.203+0000 7fcf0600d640 -1 bdev(0x563653811800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-15T23:35:33.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:33.204+0000 7fcf0600d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T23:35:36.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T23:35:36.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:36.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:36.237+0000 7feef640d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T23:35:36.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:36.237+0000 7feef640d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T23:35:36.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:36.237+0000 7feef640d640 -1 bdev(0x55ab63241800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-15T23:35:36.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:36.237+0000 7feef640d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T23:35:39.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T23:35:39.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:39.274+0000 7f6608020640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T23:35:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:39.274+0000 7f6608020640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T23:35:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:39.274+0000 7f6608020640 -1 bdev(0x55b3cff75800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-15T23:35:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:39.275+0000 7f6608020640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T23:35:42.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T23:35:42.721 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T23:35:42.752 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:42.752 INFO:tasks.ceph:role: osd.4 2024-09-15T23:35:42.752 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:35:42.752 DEBUG:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T23:35:42.823 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T23:35:42.824 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T23:35:42.824 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T23:35:42.824 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T23:35:42.824 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T23:35:42.824 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2024-09-15T23:35:42.824 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T23:35:42.824 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T23:35:42.824 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T23:35:42.824 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T23:35:42.856 INFO:teuthology.orchestra.run.smithi018.stdout:Discarding blocks...Done. 2024-09-15T23:35:42.858 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2024-09-15T23:35:42.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-15T23:35:42.931 DEBUG:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T23:35:43.003 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T23:35:43.072 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:43.072 INFO:tasks.ceph:role: osd.5 2024-09-15T23:35:43.072 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:35:43.072 DEBUG:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T23:35:43.138 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T23:35:43.170 INFO:teuthology.orchestra.run.smithi018.stdout:Discarding blocks...Done. 2024-09-15T23:35:43.172 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2024-09-15T23:35:43.172 DEBUG:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-15T23:35:43.242 DEBUG:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T23:35:43.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-15T23:35:43.377 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:43.377 INFO:tasks.ceph:role: osd.6 2024-09-15T23:35:43.377 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:35:43.377 DEBUG:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T23:35:43.445 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T23:35:43.477 INFO:teuthology.orchestra.run.smithi018.stdout:Discarding blocks...Done. 2024-09-15T23:35:43.480 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2024-09-15T23:35:43.480 DEBUG:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-15T23:35:43.550 DEBUG:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-15T23:35:43.621 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-15T23:35:43.688 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T23:35:43.689 INFO:tasks.ceph:role: osd.7 2024-09-15T23:35:43.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:35:43.689 DEBUG:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T23:35:43.754 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T23:35:43.754 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T23:35:43.755 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T23:35:43.755 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T23:35:43.755 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T23:35:43.755 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2024-09-15T23:35:43.755 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T23:35:43.755 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T23:35:43.755 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T23:35:43.755 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T23:35:43.787 INFO:teuthology.orchestra.run.smithi018.stdout:Discarding blocks...Done. 2024-09-15T23:35:43.789 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2024-09-15T23:35:43.789 DEBUG:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-15T23:35:43.857 DEBUG:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-15T23:35:43.931 DEBUG:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:44.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:44.023+0000 7f2f358d4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T23:35:44.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:44.023+0000 7f2f358d4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T23:35:44.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:44.023+0000 7f2f358d4640 -1 bdev(0x5652ab4ff800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-15T23:35:44.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:44.023+0000 7f2f358d4640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T23:35:46.944 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T23:35:46.977 DEBUG:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:47.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:47.070+0000 7f384c6f1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T23:35:47.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:47.070+0000 7f384c6f1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T23:35:47.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:47.070+0000 7f384c6f1640 -1 bdev(0x5625ca54b800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-15T23:35:47.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:47.071+0000 7f384c6f1640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T23:35:49.990 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T23:35:50.021 DEBUG:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:50.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:50.113+0000 7f90a75cc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-15T23:35:50.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:50.113+0000 7f90a75cc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-15T23:35:50.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:50.113+0000 7f90a75cc640 -1 bdev(0x55c89eab9800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-15T23:35:50.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:50.113+0000 7f90a75cc640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-15T23:35:53.522 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-15T23:35:53.552 DEBUG:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:53.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:53.644+0000 7f6b1de19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-15T23:35:53.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:53.644+0000 7f6b1de19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-15T23:35:53.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:53.644+0000 7f6b1de19640 -1 bdev(0x55acc8971800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-15T23:35:53.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:35:53.644+0000 7f6b1de19640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-15T23:35:55.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:35:55.411 DEBUG:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:35:56.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-15T23:35:56.585 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T23:35:56.585 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:56.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-15T23:35:56.616 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:56.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-15T23:35:56.685 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:56.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-15T23:35:56.753 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:56.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T23:35:56.822 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:56.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T23:35:56.891 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:56.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T23:35:56.959 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:56.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T23:35:57.026 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:57.027 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T23:35:57.057 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:57.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-15T23:35:57.128 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:57.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-15T23:35:57.196 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:57.197 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T23:35:57.264 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:57.265 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T23:35:57.333 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:57.333 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-15T23:35:57.400 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:57.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-15T23:35:57.466 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:57.466 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T23:35:57.486 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:57.486 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-15T23:35:57.545 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:57.545 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-15T23:35:57.562 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:57.562 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-15T23:35:57.619 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T23:35:57.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T23:35:57.622 DEBUG:teuthology.orchestra.run.smithi018:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T23:35:57.661 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA/b+dmb95IMxAAZ5cqDFQOkeXaL/53/u/9HA== 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA/b+dm8sQ4OhAAHNZoIaJ8fc1QsnNbpp/DaA== 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBAb+dmr3DPBRAAUbe2RHFM8HJDg4BRIDAudQ== 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBBb+dmCQ0fNhAAtZA4gUeW4XMqjUowJWFlfw== 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBFb+dmOIk9DBAAgRefS5wwcECApTOBRXFspA== 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBIb+dmUwJDDhAAUavZVwET5KkoWqm7EgEyCA== 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-09-15T23:35:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBLb+dmI8B4EBAAd1BtncP93hTVHCSHA8tOsQ== 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA/b+dmLqHhNhAA2urEMHBpz0vf+Y1IoyfwDw== 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBAb+dmCguhChAAfPnuWwKoFi6kV10uKpCwdg== 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBAb+dmwj7cERAALKBbhukiYUuNyT2l6wb6Qg== 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBQb+dmiUZxARAAyQdLywX3G5pqXWdrtFwx0A== 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBTb+dmd/ZCBBAAM7eouMiF2LtaRO8EIL/yXw== 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBWb+dm2SLSBhAA6PQkqgVM1UqzW4GJFY5Wmw== 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-09-15T23:35:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBZb+dmktF2JhAA3PMlViiCgsEKbfcuHgaohw== 2024-09-15T23:35:57.664 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-15T23:35:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBAb+dmllwaFhAA/nHwGTNRbiagVIKXM0AeYA== 2024-09-15T23:35:57.664 INFO:teuthology.orchestra.run.smithi007.stdout:[client.2] 2024-09-15T23:35:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBAb+dmn1LJGxAAaqaLtTaYKCaYfAK1JDdPcw== 2024-09-15T23:35:57.664 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-09-15T23:35:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBAb+dmFjkbHxAARK/PGgkfecGx1az8oVeB/A== 2024-09-15T23:35:57.664 INFO:teuthology.orchestra.run.smithi007.stdout:[client.3] 2024-09-15T23:35:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBAb+dm8wjKJBAAlegpp7WS85d0LtyW2elsiw== 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout:[mgr.y] 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQA/b+dmb95IMxAAZ5cqDFQOkeXaL/53/u/9HA== 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.a] 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQA/b+dm8sQ4OhAAHNZoIaJ8fc1QsnNbpp/DaA== 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.c] 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBAb+dmr3DPBRAAUbe2RHFM8HJDg4BRIDAudQ== 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.0] 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBBb+dmCQ0fNhAAtZA4gUeW4XMqjUowJWFlfw== 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.1] 2024-09-15T23:35:57.683 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBFb+dmOIk9DBAAgRefS5wwcECApTOBRXFspA== 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.2] 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBIb+dmUwJDDhAAUavZVwET5KkoWqm7EgEyCA== 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.3] 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBLb+dmI8B4EBAAd1BtncP93hTVHCSHA8tOsQ== 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout:[mgr.x] 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQA/b+dmLqHhNhAA2urEMHBpz0vf+Y1IoyfwDw== 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.b] 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBAb+dmCguhChAAfPnuWwKoFi6kV10uKpCwdg== 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.d] 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBAb+dmwj7cERAALKBbhukiYUuNyT2l6wb6Qg== 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.4] 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBQb+dmiUZxARAAyQdLywX3G5pqXWdrtFwx0A== 2024-09-15T23:35:57.684 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.5] 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBTb+dmd/ZCBBAAM7eouMiF2LtaRO8EIL/yXw== 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.6] 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBWb+dm2SLSBhAA6PQkqgVM1UqzW4GJFY5Wmw== 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.7] 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBZb+dmktF2JhAA3PMlViiCgsEKbfcuHgaohw== 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout:[client.0] 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBAb+dmllwaFhAA/nHwGTNRbiagVIKXM0AeYA== 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout:[client.2] 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBAb+dmn1LJGxAAaqaLtTaYKCaYfAK1JDdPcw== 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout:[client.1] 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBAb+dmFjkbHxAARK/PGgkfecGx1az8oVeB/A== 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout:[client.3] 2024-09-15T23:35:57.685 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQBAb+dm8wjKJBAAlegpp7WS85d0LtyW2elsiw== 2024-09-15T23:35:57.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T23:35:57.708 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T23:35:57.779 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T23:35:57.782 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T23:35:57.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T23:35:57.841 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T23:35:57.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:57.898 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:57.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:57.953 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.008 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.062 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T23:35:58.117 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T23:35:58.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T23:35:58.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T23:35:58.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T23:35:58.229 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T23:35:58.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.286 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.395 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.449 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T23:35:58.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T23:35:58.503 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T23:35:58.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T23:35:58.561 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T23:35:58.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T23:35:58.618 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T23:35:58.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T23:35:58.676 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T23:35:58.732 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T23:35:58.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T23:35:58.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T23:35:58.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T23:35:58.879 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T23:35:58.906 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T23:35:59.004 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T23:35:59.030 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T23:35:59.097 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T23:35:59.193 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T23:35:59.220 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:59.222 DEBUG:teuthology.orchestra.run.smithi018:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T23:35:59.275 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T23:35:59.276 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T23:35:59.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T23:35:59.279 INFO:tasks.ceph.mon.a:Started 2024-09-15T23:35:59.279 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T23:35:59.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T23:35:59.318 INFO:tasks.ceph.mon.b:Started 2024-09-15T23:35:59.318 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T23:35:59.318 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T23:35:59.320 INFO:tasks.ceph.mon.c:Started 2024-09-15T23:35:59.320 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T23:35:59.320 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-15T23:35:59.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-15T23:35:59.323 INFO:tasks.ceph.mgr.y:Started 2024-09-15T23:35:59.323 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T23:35:59.323 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T23:35:59.325 INFO:tasks.ceph.mgr.x:Started 2024-09-15T23:35:59.325 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T23:35:59.325 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T23:35:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.428+0000 7fe76a79f640 1 Processor -- start 2024-09-15T23:35:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.428+0000 7fe76a79f640 1 -- start start 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe76a79f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 0x7fe7640b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe76a79f640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe7640b1be0 0x7fe7641602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe76a79f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe764160050 con 0x7fe7640b2020 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe76a79f640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe7640cd620 con 0x7fe7640b1be0 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe76a79f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe76409bcf0 con 0x7fe76415fc70 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe768d3c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe7640b2020 0x7fe7640cdab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52262/0 (socket says 172.21.15.7:52262) 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe768d3c640 1 -- 172.21.15.7:0/209205768 learned_addr learned my addr 172.21.15.7:0/209205768 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe763fff640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 0x7fe7640b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe7637fe640 1 --2- 172.21.15.7:0/209205768 >> v2:172.21.15.18:3300/0 conn(0x7fe7640b1be0 0x7fe7641602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe763fff640 1 -- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 msgr2=0x7fe7640b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.429+0000 7fe763fff640 1 -- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 msgr2=0x7fe7640b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T23:35:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.430+0000 7fe763fff640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 0x7fe7640b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T23:35:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.430+0000 7fe763fff640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 0x7fe7640b37d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:35:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.430+0000 7fe7637fe640 1 -- 172.21.15.7:0/209205768 >> v2:172.21.15.18:3300/0 conn(0x7fe7640b1be0 msgr2=0x7fe7641602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T23:35:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.430+0000 7fe7637fe640 1 -- 172.21.15.7:0/209205768 >> v2:172.21.15.18:3300/0 conn(0x7fe7640b1be0 msgr2=0x7fe7641602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T23:35:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.430+0000 7fe7637fe640 1 --2- 172.21.15.7:0/209205768 >> v2:172.21.15.18:3300/0 conn(0x7fe7640b1be0 0x7fe7641602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T23:35:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.430+0000 7fe7637fe640 1 --2- 172.21.15.7:0/209205768 >> v2:172.21.15.18:3300/0 conn(0x7fe7640b1be0 0x7fe7641602c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:35:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093482647 0 0) 0x7fe764160050 con 0x7fe7640b2020 2024-09-15T23:35:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe750003520 con 0x7fe7640b2020 2024-09-15T23:35:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1395641374 0 0) 0x7fe750003520 con 0x7fe7640b2020 2024-09-15T23:35:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 msgr2=0x7fe7640b37d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:35:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 0x7fe7640b37d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 >> v2:172.21.15.18:3300/0 conn(0x7fe7640b1be0 msgr2=0x7fe7641602c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 --2- 172.21.15.7:0/209205768 >> v2:172.21.15.18:3300/0 conn(0x7fe7640b1be0 0x7fe7641602c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640b3d10 con 0x7fe7640b2020 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7fe75c0019f0 con 0x7fe7640b2020 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 >> v1:172.21.15.7:6789/0 conn(0x7fe7640b2020 legacy=0x7fe7640cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 0x7fe7500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe750003fe0 0x7fe750004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750004d40 0x7fe7500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7640cd620 con 0x7fe750004d40 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe76409bcf0 con 0x7fe750003fe0 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe75000bac0 con 0x7fe76415fc70 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe768d3c640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750004d40 0x7fe7500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:35:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.585+0000 7fe763fff640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 0x7fe7500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.586+0000 7fe7637fe640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe750003fe0 0x7fe750004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.586+0000 7fe768d3c640 1 -- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 msgr2=0x7fe7500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.586+0000 7fe768d3c640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe76415fc70 0x7fe7500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.586+0000 7fe768d3c640 1 -- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe750003fe0 msgr2=0x7fe750004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.586+0000 7fe768d3c640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe750003fe0 0x7fe750004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.586+0000 7fe768d3c640 1 -- 172.21.15.7:0/209205768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe750003520 con 0x7fe750004d40 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.586+0000 7fe768d3c640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750004d40 0x7fe7500051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0013d0 tx=0x7fe750005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848c65932891e8ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.587+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00a4e0 con 0x7fe750004d40 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.587+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c02f070 con 0x7fe750004d40 2024-09-15T23:35:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.587+0000 7fe762ffd640 1 -- 172.21.15.7:0/209205768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c02bcb0 con 0x7fe750004d40 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.587+0000 7fe76a79f640 1 -- 172.21.15.7:0/209205768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750004d40 msgr2=0x7fe7500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.587+0000 7fe76a79f640 1 --2- 172.21.15.7:0/209205768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750004d40 0x7fe7500051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0013d0 tx=0x7fe750005830 comp rx=0 tx=0).stop 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.587+0000 7fe768d3c640 1 -- 172.21.15.7:0/209205768 reap_dead start 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.587+0000 7fe76a79f640 1 -- 172.21.15.7:0/209205768 shutdown_connections 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.587+0000 7fe76a79f640 1 -- 172.21.15.7:0/209205768 >> 172.21.15.7:0/209205768 conn(0x7fe7640b3f40 msgr2=0x7fe7640b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.588+0000 7fe76a79f640 1 -- 172.21.15.7:0/209205768 shutdown_connections 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.588+0000 7fe76a79f640 1 -- 172.21.15.7:0/209205768 wait complete. 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.589+0000 7fe76a79f640 1 Processor -- start 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.589+0000 7fe76a79f640 1 -- start start 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe76a79f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe750004d40 0x7fe7640b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe76a79f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7640b7e70 0x7fe7640b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:35:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe76a79f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7640b57c0 0x7fe7640ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe76a79f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe76409ba10 con 0x7fe7640b7e70 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe76a79f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe76409b730 con 0x7fe7640b57c0 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe76a79f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe7640cd2b0 con 0x7fe750004d40 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe7637fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7640b7e70 0x7fe7640b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe7637fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7640b7e70 0x7fe7640b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42822/0 (socket says 172.21.15.7:42822) 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe7637fe640 1 -- 172.21.15.7:0/3829283590 learned_addr learned my addr 172.21.15.7:0/3829283590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe768d3c640 1 --2- 172.21.15.7:0/3829283590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7640b57c0 0x7fe7640ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe763fff640 1 --2- 172.21.15.7:0/3829283590 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe750004d40 0x7fe7640b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe7637fe640 1 -- 172.21.15.7:0/3829283590 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe750004d40 msgr2=0x7fe7640b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe7637fe640 1 --2- 172.21.15.7:0/3829283590 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe750004d40 0x7fe7640b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe7637fe640 1 -- 172.21.15.7:0/3829283590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7640b57c0 msgr2=0x7fe7640ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:35:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe7637fe640 1 --2- 172.21.15.7:0/3829283590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7640b57c0 0x7fe7640ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.590+0000 7fe7637fe640 1 -- 172.21.15.7:0/3829283590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640c96a0 con 0x7fe7640b7e70 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.591+0000 7fe7637fe640 1 --2- 172.21.15.7:0/3829283590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7640b7e70 0x7fe7640b5280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0108f0 tx=0x7fe74c010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.591+0000 7fe760ff9640 1 -- 172.21.15.7:0/3829283590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c01a020 con 0x7fe7640b7e70 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.591+0000 7fe760ff9640 1 -- 172.21.15.7:0/3829283590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c00e9c0 con 0x7fe7640b7e70 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.591+0000 7fe76a79f640 1 -- 172.21.15.7:0/3829283590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7640c9980 con 0x7fe7640b7e70 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.591+0000 7fe760ff9640 1 -- 172.21.15.7:0/3829283590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00ed90 con 0x7fe7640b7e70 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.591+0000 7fe76a79f640 1 -- 172.21.15.7:0/3829283590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7640c9f40 con 0x7fe7640b7e70 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.591+0000 7fe760ff9640 1 -- 172.21.15.7:0/3829283590 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe74c01d020 con 0x7fe7640b7e70 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.591+0000 7fe760ff9640 1 -- 172.21.15.7:0/3829283590 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe74c0148b0 con 0x7fe7640b7e70 2024-09-15T23:35:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.592+0000 7fe76a79f640 1 -- 172.21.15.7:0/3829283590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe764068d80 con 0x7fe7640b7e70 2024-09-15T23:35:59.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.594+0000 7fe760ff9640 1 -- 172.21.15.7:0/3829283590 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe764068d80 con 0x7fe7640b7e70 2024-09-15T23:35:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.658+0000 7fe76a79f640 1 -- 172.21.15.7:0/3829283590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe764160fb0 con 0x7fe7640b7e70 2024-09-15T23:35:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.660+0000 7fe760ff9640 1 -- 172.21.15.7:0/3829283590 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fe74c024020 con 0x7fe7640b7e70 2024-09-15T23:35:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-09-15T23:35:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.661+0000 7fe76a79f640 1 -- 172.21.15.7:0/3829283590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7640b7e70 msgr2=0x7fe7640b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:35:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.661+0000 7fe76a79f640 1 --2- 172.21.15.7:0/3829283590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7640b7e70 0x7fe7640b5280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0108f0 tx=0x7fe74c010db0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.661+0000 7fe76a79f640 1 -- 172.21.15.7:0/3829283590 shutdown_connections 2024-09-15T23:35:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.661+0000 7fe76a79f640 1 --2- 172.21.15.7:0/3829283590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7640b57c0 0x7fe7640ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.661+0000 7fe76a79f640 1 --2- 172.21.15.7:0/3829283590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7640b7e70 0x7fe7640b5280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.661+0000 7fe76a79f640 1 --2- 172.21.15.7:0/3829283590 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe750004d40 0x7fe7640b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:35:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.661+0000 7fe76a79f640 1 -- 172.21.15.7:0/3829283590 >> 172.21.15.7:0/3829283590 conn(0x7fe7640b3f40 msgr2=0x7fe7640b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:35:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.661+0000 7fe76a79f640 1 -- 172.21.15.7:0/3829283590 shutdown_connections 2024-09-15T23:35:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:35:59.661+0000 7fe76a79f640 1 -- 172.21.15.7:0/3829283590 wait complete. 2024-09-15T23:35:59.673 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T23:35:59.673 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T23:35:59.673 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:59.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T23:35:59.693 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:35:59.692+0000 7f57c4c0e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:35:59.699 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:59.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T23:35:59.715 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:35:59.714+0000 7fe506e1c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:35:59.736 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:35:59.736+0000 7f57c4c0e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:35:59.758 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:35:59.757+0000 7fe506e1c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:35:59.762 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:59.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T23:35:59.828 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:35:59.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T23:35:59.891 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:35:59.890+0000 7f57c4c0e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:35:59.894 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:59.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T23:35:59.905 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:35:59.904+0000 7fe506e1c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:35:59.921 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:59.921 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T23:35:59.944 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:35:59.944+0000 7f57c4c0e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:35:59.959 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:35:59.958+0000 7fe506e1c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:35:59.984 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:35:59.984 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-15T23:36:00.050 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:36:00.050 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-15T23:36:00.106 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:00.105+0000 7f57c4c0e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:36:00.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new 98ec7d46-317f-41d0-85d3-f3504f146db5 0 2024-09-15T23:36:00.150 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:00.149+0000 7fe506e1c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:36:00.222 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:00.221+0000 7f57c4c0e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:36:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.251+0000 7f4e527b9640 1 Processor -- start 2024-09-15T23:36:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.251+0000 7f4e527b9640 1 -- start start 2024-09-15T23:36:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e527b9640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e527b9640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4e4c15c620 0x7f4e4c1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e527b9640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e4c0cd150 con 0x7f4e4c067d30 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e527b9640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4e4c071290 con 0x7f4e4c15c620 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e527b9640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4c0b3cd0 con 0x7f4e4c067950 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e4bfff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e4bfff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50418/0 (socket says 172.21.15.18:50418) 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e4bfff640 1 -- 172.21.15.18:0/3290488797 learned_addr learned my addr 172.21.15.18:0/3290488797 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e4b7fe640 1 --2- 172.21.15.18:0/3290488797 >> v2:172.21.15.7:3300/0 conn(0x7f4e4c15c620 0x7f4e4c1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e4bfff640 1 -- 172.21.15.18:0/3290488797 >> v2:172.21.15.7:3300/0 conn(0x7f4e4c15c620 msgr2=0x7f4e4c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e4bfff640 1 --2- 172.21.15.18:0/3290488797 >> v2:172.21.15.7:3300/0 conn(0x7f4e4c15c620 0x7f4e4c1609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e4bfff640 1 -- 172.21.15.18:0/3290488797 >> v1:172.21.15.18:6789/0 conn(0x7f4e4c067d30 legacy=0x7f4e4c06e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e4affd640 1 -- 172.21.15.18:0/3290488797 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331252032 0 0) 0x7f4e44001490 con 0x7f4e4c067d30 2024-09-15T23:36:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.252+0000 7f4e4bfff640 1 -- 172.21.15.18:0/3290488797 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c0b3fb0 con 0x7f4e4c067950 2024-09-15T23:36:00.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.253+0000 7f4e4bfff640 1 --2- 172.21.15.18:0/3290488797 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c070c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009870 tx=0x7f4e3c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40aff472bcf4898c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.253+0000 7f4e4affd640 1 -- 172.21.15.18:0/3290488797 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c037070 con 0x7f4e4c067950 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e4affd640 1 -- 172.21.15.18:0/3290488797 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3c032440 con 0x7f4e4c067950 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e4affd640 1 -- 172.21.15.18:0/3290488797 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c032810 con 0x7f4e4c067950 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e527b9640 1 -- 172.21.15.18:0/3290488797 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 msgr2=0x7f4e4c070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e527b9640 1 --2- 172.21.15.18:0/3290488797 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c070c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009870 tx=0x7f4e3c02a270 comp rx=0 tx=0).stop 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e527b9640 1 -- 172.21.15.18:0/3290488797 shutdown_connections 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e527b9640 1 --2- 172.21.15.18:0/3290488797 >> v2:172.21.15.7:3300/0 conn(0x7f4e4c15c620 0x7f4e4c1609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e527b9640 1 --2- 172.21.15.18:0/3290488797 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c070c60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e527b9640 1 -- 172.21.15.18:0/3290488797 >> 172.21.15.18:0/3290488797 conn(0x7f4e4c072900 msgr2=0x7f4e4c072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e527b9640 1 -- 172.21.15.18:0/3290488797 shutdown_connections 2024-09-15T23:36:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.254+0000 7f4e527b9640 1 -- 172.21.15.18:0/3290488797 wait complete. 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.255+0000 7f4e527b9640 1 Processor -- start 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e527b9640 1 -- start start 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e527b9640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c09c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e527b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c067d30 0x7f4e4c09d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e527b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e4c15c620 0x7f4e4c15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e527b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c068180 con 0x7f4e4c067d30 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e527b9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c0b4570 con 0x7f4e4c15c620 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e527b9640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4c068300 con 0x7f4e4c067950 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e4bfff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c09c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e4bfff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c09c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50424/0 (socket says 172.21.15.18:50424) 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.256+0000 7f4e4bfff640 1 -- 172.21.15.18:0/1222065122 learned_addr learned my addr 172.21.15.18:0/1222065122 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e4b7fe640 1 --2- 172.21.15.18:0/1222065122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c067d30 0x7f4e4c09d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e50d56640 1 --2- 172.21.15.18:0/1222065122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e4c15c620 0x7f4e4c15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e4bfff640 1 -- 172.21.15.18:0/1222065122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e4c15c620 msgr2=0x7f4e4c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e4bfff640 1 --2- 172.21.15.18:0/1222065122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e4c15c620 0x7f4e4c15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e4bfff640 1 -- 172.21.15.18:0/1222065122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c067d30 msgr2=0x7f4e4c09d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e4bfff640 1 --2- 172.21.15.18:0/1222065122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c067d30 0x7f4e4c09d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e4bfff640 1 -- 172.21.15.18:0/1222065122 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c1f8270 con 0x7f4e4c067950 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e4bfff640 1 --2- 172.21.15.18:0/1222065122 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c09c910 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c000c00 tx=0x7f4e3c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e48ff9640 1 -- 172.21.15.18:0/1222065122 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c037040 con 0x7f4e4c067950 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e48ff9640 1 -- 172.21.15.18:0/1222065122 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3c042070 con 0x7f4e4c067950 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e48ff9640 1 -- 172.21.15.18:0/1222065122 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c030880 con 0x7f4e4c067950 2024-09-15T23:36:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e527b9640 1 -- 172.21.15.18:0/1222065122 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c1f8550 con 0x7f4e4c067950 2024-09-15T23:36:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e527b9640 1 -- 172.21.15.18:0/1222065122 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c1f8b10 con 0x7f4e4c067950 2024-09-15T23:36:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e48ff9640 1 -- 172.21.15.18:0/1222065122 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e3c030a20 con 0x7f4e4c067950 2024-09-15T23:36:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.257+0000 7f4e48ff9640 1 -- 172.21.15.18:0/1222065122 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f4e3c047bd0 con 0x7f4e4c067950 2024-09-15T23:36:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.258+0000 7f4e527b9640 1 -- 172.21.15.18:0/1222065122 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c070c60 con 0x7f4e4c067950 2024-09-15T23:36:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.262+0000 7f4e48ff9640 1 -- 172.21.15.18:0/1222065122 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4e3c047e20 con 0x7f4e4c067950 2024-09-15T23:36:00.271 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:00.270+0000 7fe506e1c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:36:00.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.333+0000 7f4e527b9640 1 -- 172.21.15.18:0/1222065122 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd new", "uuid": "98ec7d46-317f-41d0-85d3-f3504f146db5", "id": 0} v 0) -- 0x7f4e4c0b16a0 con 0x7f4e4c067950 2024-09-15T23:36:00.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.337+0000 7f4e48ff9640 1 -- 172.21.15.18:0/1222065122 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "98ec7d46-317f-41d0-85d3-f3504f146db5", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f4e3c047460 con 0x7f4e4c067950 2024-09-15T23:36:00.338 INFO:teuthology.orchestra.run.smithi018.stdout:0 2024-09-15T23:36:00.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.338+0000 7f4e527b9640 1 -- 172.21.15.18:0/1222065122 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 msgr2=0x7f4e4c09c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.338+0000 7f4e527b9640 1 --2- 172.21.15.18:0/1222065122 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c09c910 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c000c00 tx=0x7f4e3c0046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.338+0000 7f4e527b9640 1 -- 172.21.15.18:0/1222065122 shutdown_connections 2024-09-15T23:36:00.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.338+0000 7f4e527b9640 1 --2- 172.21.15.18:0/1222065122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e4c15c620 0x7f4e4c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.338+0000 7f4e527b9640 1 --2- 172.21.15.18:0/1222065122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c067d30 0x7f4e4c09d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.338+0000 7f4e527b9640 1 --2- 172.21.15.18:0/1222065122 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e4c067950 0x7f4e4c09c910 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.338+0000 7f4e527b9640 1 -- 172.21.15.18:0/1222065122 >> 172.21.15.18:0/1222065122 conn(0x7f4e4c072900 msgr2=0x7f4e4c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:00.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.338+0000 7f4e527b9640 1 -- 172.21.15.18:0/1222065122 shutdown_connections 2024-09-15T23:36:00.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.338+0000 7f4e527b9640 1 -- 172.21.15.18:0/1222065122 wait complete. 2024-09-15T23:36:00.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new 37f65ad9-e883-45f8-9600-013d978d379d 1 2024-09-15T23:36:00.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.449+0000 7f5e13f31640 1 Processor -- start 2024-09-15T23:36:00.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.449+0000 7f5e13f31640 1 -- start start 2024-09-15T23:36:00.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e13f31640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e0c15fb70 0x7f5e0c071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e13f31640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c09aa40 0x7f5e0c09ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e13f31640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0c0701d0 con 0x7f5e0c070c90 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e13f31640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5e0c070800 con 0x7f5e0c15fb70 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e13f31640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c09b3c0 con 0x7f5e0c09aa40 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e114cc640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c09aa40 0x7f5e0c09ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e124ce640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f5e0c070c90 0x7f5e0c071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:43550/0 (socket says 172.21.15.18:43550) 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e124ce640 1 -- 172.21.15.18:0/3071130164 learned_addr learned my addr 172.21.15.18:0/3071130164 (peer_addr_for_me v1:172.21.15.18:0/0) 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e11ccd640 1 --2- 172.21.15.18:0/3071130164 >> v2:172.21.15.7:3300/0 conn(0x7f5e0c15fb70 0x7f5e0c071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e10ccb640 1 -- 172.21.15.18:0/3071130164 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3946984385 0 0) 0x7f5e0c0701d0 con 0x7f5e0c070c90 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e10ccb640 1 -- 172.21.15.18:0/3071130164 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df8003520 con 0x7f5e0c070c90 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e114cc640 1 -- 172.21.15.18:0/3071130164 >> v2:172.21.15.7:3300/0 conn(0x7f5e0c15fb70 msgr2=0x7f5e0c071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e114cc640 1 --2- 172.21.15.18:0/3071130164 >> v2:172.21.15.7:3300/0 conn(0x7f5e0c15fb70 0x7f5e0c071ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e114cc640 1 -- 172.21.15.18:0/3071130164 >> v1:172.21.15.18:6789/0 conn(0x7f5e0c070c90 legacy=0x7f5e0c071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.450+0000 7f5e114cc640 1 -- 172.21.15.18:0/3071130164 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c09b780 con 0x7f5e0c09aa40 2024-09-15T23:36:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.451+0000 7f5e114cc640 1 --2- 172.21.15.18:0/3071130164 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c09aa40 0x7f5e0c09ae80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0097a0 tx=0x7f5dfc02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ff3cbe25dd6e11 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.451+0000 7f5e10ccb640 1 -- 172.21.15.18:0/3071130164 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc037070 con 0x7f5e0c09aa40 2024-09-15T23:36:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.451+0000 7f5e10ccb640 1 -- 172.21.15.18:0/3071130164 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dfc02adb0 con 0x7f5e0c09aa40 2024-09-15T23:36:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.451+0000 7f5e10ccb640 1 -- 172.21.15.18:0/3071130164 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc032660 con 0x7f5e0c09aa40 2024-09-15T23:36:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.452+0000 7f5e13f31640 1 -- 172.21.15.18:0/3071130164 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c09aa40 msgr2=0x7f5e0c09ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.452+0000 7f5e13f31640 1 --2- 172.21.15.18:0/3071130164 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c09aa40 0x7f5e0c09ae80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0097a0 tx=0x7f5dfc02a060 comp rx=0 tx=0).stop 2024-09-15T23:36:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.452+0000 7f5e13f31640 1 -- 172.21.15.18:0/3071130164 shutdown_connections 2024-09-15T23:36:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.452+0000 7f5e13f31640 1 --2- 172.21.15.18:0/3071130164 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c09aa40 0x7f5e0c09ae80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.452+0000 7f5e13f31640 1 --2- 172.21.15.18:0/3071130164 >> v2:172.21.15.7:3300/0 conn(0x7f5e0c15fb70 0x7f5e0c071ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.452+0000 7f5e13f31640 1 -- 172.21.15.18:0/3071130164 >> 172.21.15.18:0/3071130164 conn(0x7f5e0c0b3ef0 msgr2=0x7f5e0c0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.452+0000 7f5e13f31640 1 -- 172.21.15.18:0/3071130164 shutdown_connections 2024-09-15T23:36:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.452+0000 7f5e13f31640 1 -- 172.21.15.18:0/3071130164 wait complete. 2024-09-15T23:36:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.453+0000 7f5e13f31640 1 Processor -- start 2024-09-15T23:36:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.453+0000 7f5e13f31640 1 -- start start 2024-09-15T23:36:00.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e13f31640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c070c90 0x7f5e0c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e13f31640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e0c09aa40 0x7f5e0c15e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e13f31640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0c15fb70 0x7f5e0c0c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e13f31640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c070490 con 0x7f5e0c15fb70 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e13f31640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c09bd40 con 0x7f5e0c09aa40 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e13f31640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c15ff50 con 0x7f5e0c070c90 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e114cc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e0c09aa40 0x7f5e0c15e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e11ccd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c070c90 0x7f5e0c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e114cc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e0c09aa40 0x7f5e0c15e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53122/0 (socket says 172.21.15.18:53122) 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e114cc640 1 -- 172.21.15.18:0/1681196198 learned_addr learned my addr 172.21.15.18:0/1681196198 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e124ce640 1 --2- 172.21.15.18:0/1681196198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0c15fb70 0x7f5e0c0c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e114cc640 1 -- 172.21.15.18:0/1681196198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c070c90 msgr2=0x7f5e0c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e114cc640 1 --2- 172.21.15.18:0/1681196198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c070c90 0x7f5e0c0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e114cc640 1 -- 172.21.15.18:0/1681196198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0c15fb70 msgr2=0x7f5e0c0c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e114cc640 1 --2- 172.21.15.18:0/1681196198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0c15fb70 0x7f5e0c0c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e114cc640 1 -- 172.21.15.18:0/1681196198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c0c7af0 con 0x7f5e0c09aa40 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.454+0000 7f5e11ccd640 1 --2- 172.21.15.18:0/1681196198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c070c90 0x7f5e0c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.455+0000 7f5e114cc640 1 --2- 172.21.15.18:0/1681196198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e0c09aa40 0x7f5e0c15e460 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0098c0 tx=0x7f5dfc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.455+0000 7f5e027fc640 1 -- 172.21.15.18:0/1681196198 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc041030 con 0x7f5e0c09aa40 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.455+0000 7f5e027fc640 1 -- 172.21.15.18:0/1681196198 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dfc03c670 con 0x7f5e0c09aa40 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.455+0000 7f5e13f31640 1 -- 172.21.15.18:0/1681196198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c0c7d90 con 0x7f5e0c09aa40 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.455+0000 7f5e027fc640 1 -- 172.21.15.18:0/1681196198 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc037040 con 0x7f5e0c09aa40 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.455+0000 7f5e13f31640 1 -- 172.21.15.18:0/1681196198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c0c8070 con 0x7f5e0c09aa40 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.455+0000 7f5e027fc640 1 -- 172.21.15.18:0/1681196198 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5dfc044070 con 0x7f5e0c09aa40 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.455+0000 7f5e027fc640 1 -- 172.21.15.18:0/1681196198 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f5dfc046b60 con 0x7f5e0c09aa40 2024-09-15T23:36:00.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.456+0000 7f5e13f31640 1 -- 172.21.15.18:0/1681196198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c09ae80 con 0x7f5e0c09aa40 2024-09-15T23:36:00.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.458+0000 7f5e027fc640 1 -- 172.21.15.18:0/1681196198 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5dfc046db0 con 0x7f5e0c09aa40 2024-09-15T23:36:00.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.528+0000 7f5e13f31640 1 -- 172.21.15.18:0/1681196198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "37f65ad9-e883-45f8-9600-013d978d379d", "id": 1} v 0) -- 0x7f5e0c0b4c50 con 0x7f5e0c09aa40 2024-09-15T23:36:00.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.531+0000 7f5e027fc640 1 -- 172.21.15.18:0/1681196198 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37f65ad9-e883-45f8-9600-013d978d379d", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f5dfc02ac70 con 0x7f5e0c09aa40 2024-09-15T23:36:00.532 INFO:teuthology.orchestra.run.smithi018.stdout:1 2024-09-15T23:36:00.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.532+0000 7f5e13f31640 1 -- 172.21.15.18:0/1681196198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e0c09aa40 msgr2=0x7f5e0c15e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.532+0000 7f5e13f31640 1 --2- 172.21.15.18:0/1681196198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e0c09aa40 0x7f5e0c15e460 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0098c0 tx=0x7f5dfc0046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.532+0000 7f5e13f31640 1 -- 172.21.15.18:0/1681196198 shutdown_connections 2024-09-15T23:36:00.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.532+0000 7f5e13f31640 1 --2- 172.21.15.18:0/1681196198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0c15fb70 0x7f5e0c0c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.532+0000 7f5e13f31640 1 --2- 172.21.15.18:0/1681196198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e0c09aa40 0x7f5e0c15e460 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.532+0000 7f5e13f31640 1 --2- 172.21.15.18:0/1681196198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e0c070c90 0x7f5e0c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.532+0000 7f5e13f31640 1 -- 172.21.15.18:0/1681196198 >> 172.21.15.18:0/1681196198 conn(0x7f5e0c0b3ef0 msgr2=0x7f5e0c160240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:00.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.532+0000 7f5e13f31640 1 -- 172.21.15.18:0/1681196198 shutdown_connections 2024-09-15T23:36:00.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.532+0000 7f5e13f31640 1 -- 172.21.15.18:0/1681196198 wait complete. 2024-09-15T23:36:00.543 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new 31daf586-306d-4234-bc21-cf9833bf3af6 2 2024-09-15T23:36:00.568 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:00.567+0000 7f57c4c0e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:36:00.621 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:00.620+0000 7f57c4c0e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:36:00.623 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:00.622+0000 7fe506e1c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:36:00.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.643+0000 7f7fc3966640 1 Processor -- start 2024-09-15T23:36:00.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.643+0000 7f7fc3966640 1 -- start start 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc3966640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7fbc0b3050 0x7f7fbc0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc3966640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc09ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc3966640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fbc0b4e40 con 0x7f7fbc162040 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc3966640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc06df40 con 0x7f7fbc0b3050 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc3966640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc06e220 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc0f01640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7fbc0b3050 0x7f7fbc0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc1f03640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc09ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc0f01640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7fbc0b3050 0x7f7fbc0b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53128/0 (socket says 172.21.15.18:53128) 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc1f03640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc09ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50444/0 (socket says 172.21.15.18:50444) 2024-09-15T23:36:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.644+0000 7f7fc0f01640 1 -- 172.21.15.18:0/355095315 learned_addr learned my addr 172.21.15.18:0/355095315 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:00.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.645+0000 7f7fc1f03640 1 -- 172.21.15.18:0/355095315 >> v2:172.21.15.18:3300/0 conn(0x7f7fbc0b3050 msgr2=0x7f7fbc0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:00.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.645+0000 7f7fc1f03640 1 --2- 172.21.15.18:0/355095315 >> v2:172.21.15.18:3300/0 conn(0x7f7fbc0b3050 0x7f7fbc0b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.645+0000 7f7fc1f03640 1 -- 172.21.15.18:0/355095315 >> v1:172.21.15.7:6789/0 conn(0x7f7fbc162040 legacy=0x7f7fbc0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.645+0000 7f7fc1f03640 1 -- 172.21.15.18:0/355095315 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc06e500 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.645+0000 7f7fc1f03640 1 --2- 172.21.15.18:0/355095315 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc09ade0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009d90 tx=0x7f7fb402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ae3d6bdbcc05eb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:00.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.645+0000 7f7faffff640 1 -- 172.21.15.18:0/355095315 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4007740 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7faffff640 1 -- 172.21.15.18:0/355095315 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb40078e0 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7faffff640 1 -- 172.21.15.18:0/355095315 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4007cb0 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7fc3966640 1 -- 172.21.15.18:0/355095315 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 msgr2=0x7f7fbc09ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7fc3966640 1 --2- 172.21.15.18:0/355095315 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc09ade0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009d90 tx=0x7f7fb402a660 comp rx=0 tx=0).stop 2024-09-15T23:36:00.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7fc3966640 1 -- 172.21.15.18:0/355095315 shutdown_connections 2024-09-15T23:36:00.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7fc3966640 1 --2- 172.21.15.18:0/355095315 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc09ade0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7fc3966640 1 --2- 172.21.15.18:0/355095315 >> v2:172.21.15.18:3300/0 conn(0x7f7fbc0b3050 0x7f7fbc0b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7fc3966640 1 -- 172.21.15.18:0/355095315 >> 172.21.15.18:0/355095315 conn(0x7f7fbc0b3f40 msgr2=0x7f7fbc0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:00.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7fc3966640 1 -- 172.21.15.18:0/355095315 shutdown_connections 2024-09-15T23:36:00.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.646+0000 7f7fc3966640 1 -- 172.21.15.18:0/355095315 wait complete. 2024-09-15T23:36:00.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.647+0000 7f7fc3966640 1 Processor -- start 2024-09-15T23:36:00.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc3966640 1 -- start start 2024-09-15T23:36:00.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc3966640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fbc0b3050 0x7f7fbc0b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc3966640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc0cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc3966640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc162040 0x7f7fbc15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc3966640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0b4be0 con 0x7f7fbc0b3050 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc3966640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0b4a60 con 0x7f7fbc162040 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc3966640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc09b320 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc0f01640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc0cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc1f03640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc162040 0x7f7fbc15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc1f03640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc162040 0x7f7fbc15e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53132/0 (socket says 172.21.15.18:53132) 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc1f03640 1 -- 172.21.15.18:0/2935373680 learned_addr learned my addr 172.21.15.18:0/2935373680 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc0f01640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc0cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50450/0 (socket says 172.21.15.18:50450) 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.648+0000 7f7fc1702640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fbc0b3050 0x7f7fbc0b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc0f01640 1 -- 172.21.15.18:0/2935373680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc162040 msgr2=0x7f7fbc15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc0f01640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc162040 0x7f7fbc15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc0f01640 1 -- 172.21.15.18:0/2935373680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fbc0b3050 msgr2=0x7f7fbc0b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc0f01640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fbc0b3050 0x7f7fbc0b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc0f01640 1 -- 172.21.15.18:0/2935373680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc0cd1f0 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc1702640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fbc0b3050 0x7f7fbc0b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc1f03640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc162040 0x7f7fbc15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc0f01640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc0cca80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800e850 tx=0x7f7fb800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fadffb640 1 -- 172.21.15.18:0/2935373680 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8017070 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fadffb640 1 -- 172.21.15.18:0/2935373680 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb800c940 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc3966640 1 -- 172.21.15.18:0/2935373680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc0cd4d0 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fadffb640 1 -- 172.21.15.18:0/2935373680 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb800ccf0 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.649+0000 7f7fc3966640 1 -- 172.21.15.18:0/2935373680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc0cda90 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.650+0000 7f7fadffb640 1 -- 172.21.15.18:0/2935373680 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7fb801a070 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.650+0000 7f7fadffb640 1 -- 172.21.15.18:0/2935373680 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f7fb801c870 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.650+0000 7f7fc3966640 1 -- 172.21.15.18:0/2935373680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc06cd20 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.652+0000 7f7fadffb640 1 -- 172.21.15.18:0/2935373680 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7fbc06cd20 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.677 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:00.676+0000 7fe506e1c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:36:00.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.724+0000 7f7fc3966640 1 -- 172.21.15.18:0/2935373680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd new", "uuid": "31daf586-306d-4234-bc21-cf9833bf3af6", "id": 2} v 0) -- 0x7f7fbc068d80 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.727+0000 7f7fadffb640 1 -- 172.21.15.18:0/2935373680 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "31daf586-306d-4234-bc21-cf9833bf3af6", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f7fbc068d80 con 0x7f7fbc0d0e90 2024-09-15T23:36:00.728 INFO:teuthology.orchestra.run.smithi018.stdout:2 2024-09-15T23:36:00.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.728+0000 7f7fc3966640 1 -- 172.21.15.18:0/2935373680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 msgr2=0x7f7fbc0cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.728+0000 7f7fc3966640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc0cca80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800e850 tx=0x7f7fb800ed10 comp rx=0 tx=0).stop 2024-09-15T23:36:00.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.728+0000 7f7fc3966640 1 -- 172.21.15.18:0/2935373680 shutdown_connections 2024-09-15T23:36:00.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.728+0000 7f7fc3966640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc162040 0x7f7fbc15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.728+0000 7f7fc3966640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7fbc0d0e90 0x7f7fbc0cca80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.728+0000 7f7fc3966640 1 --2- 172.21.15.18:0/2935373680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fbc0b3050 0x7f7fbc0b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.728+0000 7f7fc3966640 1 -- 172.21.15.18:0/2935373680 >> 172.21.15.18:0/2935373680 conn(0x7f7fbc0b3f40 msgr2=0x7f7fbc15f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:00.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.728+0000 7f7fc3966640 1 -- 172.21.15.18:0/2935373680 shutdown_connections 2024-09-15T23:36:00.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.728+0000 7f7fc3966640 1 -- 172.21.15.18:0/2935373680 wait complete. 2024-09-15T23:36:00.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new 8a976294-6121-4529-8794-f62609f9d76f 3 2024-09-15T23:36:00.778 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:00.777+0000 7f57c4c0e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:36:00.839 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:00.838+0000 7fe506e1c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:36:00.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.840+0000 7f3ba411a640 1 Processor -- start 2024-09-15T23:36:00.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.840+0000 7f3ba411a640 1 -- start start 2024-09-15T23:36:00.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba411a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c15b270 0x7f3b9c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba411a640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3b9c0b54e0 0x7f3b9c09b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba411a640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b9c15de30 con 0x7f3b9c0b47d0 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba411a640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9c15e110 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba411a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c15e3f0 con 0x7f3b9c15b270 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba1eb6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c15b270 0x7f3b9c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba26b7640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3b9c0b54e0 0x7f3b9c09b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba26b7640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3b9c0b54e0 0x7f3b9c09b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53144/0 (socket says 172.21.15.18:53144) 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba1eb6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c15b270 0x7f3b9c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50462/0 (socket says 172.21.15.18:50462) 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba26b7640 1 -- 172.21.15.18:0/2303513488 learned_addr learned my addr 172.21.15.18:0/2303513488 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba1eb6640 1 -- 172.21.15.18:0/2303513488 >> v2:172.21.15.18:3300/0 conn(0x7f3b9c0b54e0 msgr2=0x7f3b9c09b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba1eb6640 1 --2- 172.21.15.18:0/2303513488 >> v2:172.21.15.18:3300/0 conn(0x7f3b9c0b54e0 0x7f3b9c09b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba1eb6640 1 -- 172.21.15.18:0/2303513488 >> v1:172.21.15.7:6789/0 conn(0x7f3b9c0b47d0 legacy=0x7f3b9c0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba1eb6640 1 -- 172.21.15.18:0/2303513488 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c15e6d0 con 0x7f3b9c15b270 2024-09-15T23:36:00.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.841+0000 7f3ba26b7640 1 --2- 172.21.15.18:0/2303513488 >> v2:172.21.15.18:3300/0 conn(0x7f3b9c0b54e0 0x7f3b9c09b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.842+0000 7f3ba1eb6640 1 --2- 172.21.15.18:0/2303513488 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c15b270 0x7f3b9c15d4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3b980097a0 tx=0x7f3b9802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23cd1609cb8fe344 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.842+0000 7f3ba0eb4640 1 -- 172.21.15.18:0/2303513488 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98037070 con 0x7f3b9c15b270 2024-09-15T23:36:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.842+0000 7f3ba0eb4640 1 -- 172.21.15.18:0/2303513488 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b98032440 con 0x7f3b9c15b270 2024-09-15T23:36:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.842+0000 7f3ba0eb4640 1 -- 172.21.15.18:0/2303513488 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98032810 con 0x7f3b9c15b270 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.843+0000 7f3ba411a640 1 -- 172.21.15.18:0/2303513488 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c15b270 msgr2=0x7f3b9c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.843+0000 7f3ba411a640 1 --2- 172.21.15.18:0/2303513488 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c15b270 0x7f3b9c15d4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3b980097a0 tx=0x7f3b9802a180 comp rx=0 tx=0).stop 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.843+0000 7f3ba411a640 1 -- 172.21.15.18:0/2303513488 shutdown_connections 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.843+0000 7f3ba411a640 1 --2- 172.21.15.18:0/2303513488 >> v2:172.21.15.18:3300/0 conn(0x7f3b9c0b54e0 0x7f3b9c09b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.843+0000 7f3ba411a640 1 --2- 172.21.15.18:0/2303513488 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c15b270 0x7f3b9c15d4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.843+0000 7f3ba411a640 1 -- 172.21.15.18:0/2303513488 >> 172.21.15.18:0/2303513488 conn(0x7f3b9c0b67e0 msgr2=0x7f3b9c0b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.843+0000 7f3ba411a640 1 -- 172.21.15.18:0/2303513488 shutdown_connections 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.843+0000 7f3ba411a640 1 -- 172.21.15.18:0/2303513488 wait complete. 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.844+0000 7f3ba411a640 1 Processor -- start 2024-09-15T23:36:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.844+0000 7f3ba411a640 1 -- start start 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba411a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c0b47d0 0x7f3b9c1510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba411a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b9c0b54e0 0x7f3b9c151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba411a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b9c15b270 0x7f3b9c151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba411a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c15c7e0 con 0x7f3b9c15b270 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba411a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c15c660 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba411a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c15da00 con 0x7f3b9c0b47d0 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba1eb6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c0b47d0 0x7f3b9c1510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba16b5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b9c0b54e0 0x7f3b9c151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba1eb6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c0b47d0 0x7f3b9c1510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50468/0 (socket says 172.21.15.18:50468) 2024-09-15T23:36:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba1eb6640 1 -- 172.21.15.18:0/596776137 learned_addr learned my addr 172.21.15.18:0/596776137 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba26b7640 1 --2- 172.21.15.18:0/596776137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b9c15b270 0x7f3b9c151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba16b5640 1 -- 172.21.15.18:0/596776137 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c0b47d0 msgr2=0x7f3b9c1510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba16b5640 1 --2- 172.21.15.18:0/596776137 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c0b47d0 0x7f3b9c1510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba16b5640 1 -- 172.21.15.18:0/596776137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b9c15b270 msgr2=0x7f3b9c151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba16b5640 1 --2- 172.21.15.18:0/596776137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b9c15b270 0x7f3b9c151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba16b5640 1 -- 172.21.15.18:0/596776137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c1515f0 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba1eb6640 1 --2- 172.21.15.18:0/596776137 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c0b47d0 0x7f3b9c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.845+0000 7f3ba16b5640 1 --2- 172.21.15.18:0/596776137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b9c0b54e0 0x7f3b9c151900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b2f90 tx=0x7f3b8c00c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.846+0000 7f3b927fc640 1 -- 172.21.15.18:0/596776137 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0052d0 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.846+0000 7f3b927fc640 1 -- 172.21.15.18:0/596776137 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b8c005490 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.846+0000 7f3ba411a640 1 -- 172.21.15.18:0/596776137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9c09c4a0 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.846+0000 7f3b927fc640 1 -- 172.21.15.18:0/596776137 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c007260 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.846+0000 7f3ba411a640 1 -- 172.21.15.18:0/596776137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9c09ca60 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.846+0000 7f3b927fc640 1 -- 172.21.15.18:0/596776137 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b8c007400 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.846+0000 7f3b927fc640 1 -- 172.21.15.18:0/596776137 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f3b8c011590 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.847+0000 7f3ba411a640 1 -- 172.21.15.18:0/596776137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9c068e90 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.849+0000 7f3b927fc640 1 -- 172.21.15.18:0/596776137 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3b9c068e90 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.919+0000 7f3ba411a640 1 -- 172.21.15.18:0/596776137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8a976294-6121-4529-8794-f62609f9d76f", "id": 3} v 0) -- 0x7f3b9c09e840 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.922+0000 7f3b927fc640 1 -- 172.21.15.18:0/596776137 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8a976294-6121-4529-8794-f62609f9d76f", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f3b8c003a50 con 0x7f3b9c0b54e0 2024-09-15T23:36:00.923 INFO:teuthology.orchestra.run.smithi018.stdout:3 2024-09-15T23:36:00.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.923+0000 7f3ba411a640 1 -- 172.21.15.18:0/596776137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b9c0b54e0 msgr2=0x7f3b9c151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:00.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.923+0000 7f3ba411a640 1 --2- 172.21.15.18:0/596776137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b9c0b54e0 0x7f3b9c151900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b2f90 tx=0x7f3b8c00c7a0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.923+0000 7f3ba411a640 1 -- 172.21.15.18:0/596776137 shutdown_connections 2024-09-15T23:36:00.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.923+0000 7f3ba411a640 1 --2- 172.21.15.18:0/596776137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b9c15b270 0x7f3b9c151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.923+0000 7f3ba411a640 1 --2- 172.21.15.18:0/596776137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b9c0b54e0 0x7f3b9c151900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.923+0000 7f3ba411a640 1 --2- 172.21.15.18:0/596776137 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b9c0b47d0 0x7f3b9c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:00.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.923+0000 7f3ba411a640 1 -- 172.21.15.18:0/596776137 >> 172.21.15.18:0/596776137 conn(0x7f3b9c0b67e0 msgr2=0x7f3b9c0d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:00.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.923+0000 7f3ba411a640 1 -- 172.21.15.18:0/596776137 shutdown_connections 2024-09-15T23:36:00.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:00.923+0000 7f3ba411a640 1 -- 172.21.15.18:0/596776137 wait complete. 2024-09-15T23:36:00.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1 4 2024-09-15T23:36:01.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.035+0000 7f96e16e4640 1 Processor -- start 2024-09-15T23:36:01.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.035+0000 7f96e16e4640 1 -- start start 2024-09-15T23:36:01.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.036+0000 7f96e16e4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f96dc0b1be0 0x7f96dc1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.036+0000 7f96e16e4640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96dc160050 con 0x7f96dc0b2050 2024-09-15T23:36:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.036+0000 7f96e16e4640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96dc09bcf0 con 0x7f96dc15fc70 2024-09-15T23:36:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.036+0000 7f96e16e4640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f96dc0cd620 con 0x7f96dc0b1be0 2024-09-15T23:36:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.036+0000 7f96da7fc640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f96dc0b1be0 0x7f96dc1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.036+0000 7f96da7fc640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f96dc0b1be0 0x7f96dc1602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53164/0 (socket says 172.21.15.18:53164) 2024-09-15T23:36:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.036+0000 7f96da7fc640 1 -- 172.21.15.18:0/1780698840 learned_addr learned my addr 172.21.15.18:0/1780698840 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.036+0000 7f96daffd640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f96dc15fc70 0x7f96dc0b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:43556/0 (socket says 172.21.15.18:43556) 2024-09-15T23:36:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.037+0000 7f96da7fc640 1 -- 172.21.15.18:0/1780698840 >> v1:172.21.15.18:6789/0 conn(0x7f96dc15fc70 legacy=0x7f96dc0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.037+0000 7f96da7fc640 1 -- 172.21.15.18:0/1780698840 >> v1:172.21.15.7:6789/0 conn(0x7f96dc0b2050 legacy=0x7f96dc0cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.037+0000 7f96da7fc640 1 -- 172.21.15.18:0/1780698840 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc09b450 con 0x7f96dc0b1be0 2024-09-15T23:36:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.037+0000 7f96da7fc640 1 --2- 172.21.15.18:0/1780698840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b1be0 0x7f96dc1602c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009870 tx=0x7f96d002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ef8bf991df404d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.037+0000 7f96d9ffb640 1 -- 172.21.15.18:0/1780698840 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0037070 con 0x7f96dc0b1be0 2024-09-15T23:36:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.037+0000 7f96d9ffb640 1 -- 172.21.15.18:0/1780698840 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d002ae50 con 0x7f96dc0b1be0 2024-09-15T23:36:01.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.037+0000 7f96d9ffb640 1 -- 172.21.15.18:0/1780698840 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0032860 con 0x7f96dc0b1be0 2024-09-15T23:36:01.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.038+0000 7f96e16e4640 1 -- 172.21.15.18:0/1780698840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b1be0 msgr2=0x7f96dc1602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.038+0000 7f96e16e4640 1 --2- 172.21.15.18:0/1780698840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b1be0 0x7f96dc1602c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009870 tx=0x7f96d002a300 comp rx=0 tx=0).stop 2024-09-15T23:36:01.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.038+0000 7f96e16e4640 1 -- 172.21.15.18:0/1780698840 shutdown_connections 2024-09-15T23:36:01.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.038+0000 7f96e16e4640 1 --2- 172.21.15.18:0/1780698840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b1be0 0x7f96dc1602c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.038+0000 7f96e16e4640 1 -- 172.21.15.18:0/1780698840 >> 172.21.15.18:0/1780698840 conn(0x7f96dc0b3f40 msgr2=0x7f96dc0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.038+0000 7f96e16e4640 1 -- 172.21.15.18:0/1780698840 shutdown_connections 2024-09-15T23:36:01.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.038+0000 7f96e16e4640 1 -- 172.21.15.18:0/1780698840 wait complete. 2024-09-15T23:36:01.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.039+0000 7f96e16e4640 1 Processor -- start 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.039+0000 7f96e16e4640 1 -- start start 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96e16e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc0b1be0 0x7f96dc0b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96e16e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b2050 0x7f96dc1506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96e16e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96dc15fc70 0x7f96dc0c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96e16e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0cd2b0 con 0x7f96dc0b1be0 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96e16e4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc09ba10 con 0x7f96dc0b2050 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96e16e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc160fb0 con 0x7f96dc15fc70 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b2050 0x7f96dc1506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b2050 0x7f96dc1506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53170/0 (socket says 172.21.15.18:53170) 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96daffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc0b1be0 0x7f96dc0b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96da7fc640 1 -- 172.21.15.18:0/899502259 learned_addr learned my addr 172.21.15.18:0/899502259 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:01.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.040+0000 7f96db7fe640 1 --2- 172.21.15.18:0/899502259 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96dc15fc70 0x7f96dc0c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96da7fc640 1 -- 172.21.15.18:0/899502259 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96dc15fc70 msgr2=0x7f96dc0c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96da7fc640 1 --2- 172.21.15.18:0/899502259 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96dc15fc70 0x7f96dc0c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96da7fc640 1 -- 172.21.15.18:0/899502259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc0b1be0 msgr2=0x7f96dc0b7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96da7fc640 1 --2- 172.21.15.18:0/899502259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc0b1be0 0x7f96dc0b7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96da7fc640 1 -- 172.21.15.18:0/899502259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc0c8190 con 0x7f96dc0b2050 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96daffd640 1 --2- 172.21.15.18:0/899502259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc0b1be0 0x7f96dc0b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96da7fc640 1 --2- 172.21.15.18:0/899502259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b2050 0x7f96dc1506a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f96d002a340 tx=0x7f96d00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96c37fe640 1 -- 172.21.15.18:0/899502259 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0041020 con 0x7f96dc0b2050 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96c37fe640 1 -- 172.21.15.18:0/899502259 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d003c890 con 0x7f96dc0b2050 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96e16e4640 1 -- 172.21.15.18:0/899502259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc0c8470 con 0x7f96dc0b2050 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96c37fe640 1 -- 172.21.15.18:0/899502259 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0037040 con 0x7f96dc0b2050 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96e16e4640 1 -- 172.21.15.18:0/899502259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc0c6ee0 con 0x7f96dc0b2050 2024-09-15T23:36:01.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96c37fe640 1 -- 172.21.15.18:0/899502259 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f96d0044020 con 0x7f96dc0b2050 2024-09-15T23:36:01.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.041+0000 7f96c37fe640 1 -- 172.21.15.18:0/899502259 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f96d003dc80 con 0x7f96dc0b2050 2024-09-15T23:36:01.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.042+0000 7f96e16e4640 1 -- 172.21.15.18:0/899502259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc0b37d0 con 0x7f96dc0b2050 2024-09-15T23:36:01.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.044+0000 7f96c37fe640 1 -- 172.21.15.18:0/899502259 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f96d004b030 con 0x7f96dc0b2050 2024-09-15T23:36:01.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.115+0000 7f96e16e4640 1 -- 172.21.15.18:0/899502259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1", "id": 4} v 0) -- 0x7f96dc0cdd00 con 0x7f96dc0b2050 2024-09-15T23:36:01.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.118+0000 7f96c37fe640 1 -- 172.21.15.18:0/899502259 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f96d003cd50 con 0x7f96dc0b2050 2024-09-15T23:36:01.119 INFO:teuthology.orchestra.run.smithi018.stdout:4 2024-09-15T23:36:01.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.119+0000 7f96e16e4640 1 -- 172.21.15.18:0/899502259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b2050 msgr2=0x7f96dc1506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.119+0000 7f96e16e4640 1 --2- 172.21.15.18:0/899502259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b2050 0x7f96dc1506a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f96d002a340 tx=0x7f96d00046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.119+0000 7f96e16e4640 1 -- 172.21.15.18:0/899502259 shutdown_connections 2024-09-15T23:36:01.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.119+0000 7f96e16e4640 1 --2- 172.21.15.18:0/899502259 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96dc15fc70 0x7f96dc0c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.119+0000 7f96e16e4640 1 --2- 172.21.15.18:0/899502259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96dc0b2050 0x7f96dc1506a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.119+0000 7f96e16e4640 1 --2- 172.21.15.18:0/899502259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc0b1be0 0x7f96dc0b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.119+0000 7f96e16e4640 1 -- 172.21.15.18:0/899502259 >> 172.21.15.18:0/899502259 conn(0x7f96dc0b3f40 msgr2=0x7f96dc15aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.119+0000 7f96e16e4640 1 -- 172.21.15.18:0/899502259 shutdown_connections 2024-09-15T23:36:01.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.119+0000 7f96e16e4640 1 -- 172.21.15.18:0/899502259 wait complete. 2024-09-15T23:36:01.130 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new a88ba428-0477-4d56-8889-552e0b7229ee 5 2024-09-15T23:36:01.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.231+0000 7f8b67ca5640 1 Processor -- start 2024-09-15T23:36:01.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.231+0000 7f8b67ca5640 1 -- start start 2024-09-15T23:36:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b67ca5640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8b6006c320 0x7f8b60073520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b67ca5640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8b600b60a0 0x7f8b60160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b67ca5640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b600b65d0 con 0x7f8b6015e880 2024-09-15T23:36:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b67ca5640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8b6015fc70 con 0x7f8b600b60a0 2024-09-15T23:36:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b67ca5640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8b601603b0 con 0x7f8b6006c320 2024-09-15T23:36:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b65240640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8b6006c320 0x7f8b60073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b65a41640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f8b6015e880 0x7f8b6006bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:43566/0 (socket says 172.21.15.18:43566) 2024-09-15T23:36:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b65240640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8b6006c320 0x7f8b60073520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53174/0 (socket says 172.21.15.18:53174) 2024-09-15T23:36:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b65a41640 1 -- 172.21.15.18:0/1084099063 learned_addr learned my addr 172.21.15.18:0/1084099063 (peer_addr_for_me v1:172.21.15.18:0/0) 2024-09-15T23:36:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b66242640 1 --2- 172.21.15.18:0/1084099063 >> v2:172.21.15.7:3300/0 conn(0x7f8b600b60a0 0x7f8b60160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b65240640 1 -- 172.21.15.18:0/1084099063 >> v2:172.21.15.7:3300/0 conn(0x7f8b600b60a0 msgr2=0x7f8b60160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b65240640 1 --2- 172.21.15.18:0/1084099063 >> v2:172.21.15.7:3300/0 conn(0x7f8b600b60a0 0x7f8b60160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.232+0000 7f8b65240640 1 -- 172.21.15.18:0/1084099063 >> v1:172.21.15.18:6789/0 conn(0x7f8b6015e880 legacy=0x7f8b6006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.233+0000 7f8b64a3f640 1 -- 172.21.15.18:0/1084099063 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 408529118 0 0) 0x7f8b600b65d0 con 0x7f8b6015e880 2024-09-15T23:36:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.233+0000 7f8b65240640 1 -- 172.21.15.18:0/1084099063 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b60160690 con 0x7f8b6006c320 2024-09-15T23:36:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.233+0000 7f8b66242640 1 --2- 172.21.15.18:0/1084099063 >> v2:172.21.15.7:3300/0 conn(0x7f8b600b60a0 0x7f8b60160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:01.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.234+0000 7f8b65240640 1 --2- 172.21.15.18:0/1084099063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6006c320 0x7f8b60073520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8b600b0e10 tx=0x7f8b5002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=399ebeab63759e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.234+0000 7f8b64a3f640 1 -- 172.21.15.18:0/1084099063 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50037070 con 0x7f8b6006c320 2024-09-15T23:36:01.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.234+0000 7f8b64a3f640 1 -- 172.21.15.18:0/1084099063 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b50032440 con 0x7f8b6006c320 2024-09-15T23:36:01.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.234+0000 7f8b64a3f640 1 -- 172.21.15.18:0/1084099063 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50032810 con 0x7f8b6006c320 2024-09-15T23:36:01.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.234+0000 7f8b67ca5640 1 -- 172.21.15.18:0/1084099063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6006c320 msgr2=0x7f8b60073520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.234+0000 7f8b67ca5640 1 --2- 172.21.15.18:0/1084099063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6006c320 0x7f8b60073520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8b600b0e10 tx=0x7f8b5002a1a0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.235+0000 7f8b67ca5640 1 -- 172.21.15.18:0/1084099063 shutdown_connections 2024-09-15T23:36:01.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.235+0000 7f8b67ca5640 1 --2- 172.21.15.18:0/1084099063 >> v2:172.21.15.7:3300/0 conn(0x7f8b600b60a0 0x7f8b60160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.235+0000 7f8b67ca5640 1 --2- 172.21.15.18:0/1084099063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6006c320 0x7f8b60073520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.235+0000 7f8b67ca5640 1 -- 172.21.15.18:0/1084099063 >> 172.21.15.18:0/1084099063 conn(0x7f8b600b6a90 msgr2=0x7f8b600b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.235+0000 7f8b67ca5640 1 -- 172.21.15.18:0/1084099063 shutdown_connections 2024-09-15T23:36:01.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.235+0000 7f8b67ca5640 1 -- 172.21.15.18:0/1084099063 wait complete. 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.236+0000 7f8b67ca5640 1 Processor -- start 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.236+0000 7f8b67ca5640 1 -- start start 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b67ca5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b6006c320 0x7f8b600ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b67ca5640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b600b60a0 0x7f8b6014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b67ca5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6015e880 0x7f8b6006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b67ca5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6006ca30 con 0x7f8b6006c320 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b67ca5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6006c8b0 con 0x7f8b6015e880 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b67ca5640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8b601613d0 con 0x7f8b600b60a0 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65240640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b600b60a0 0x7f8b6014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b66242640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6015e880 0x7f8b6006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65240640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b600b60a0 0x7f8b6014de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50484/0 (socket says 172.21.15.18:50484) 2024-09-15T23:36:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b66242640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6015e880 0x7f8b6006f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53184/0 (socket says 172.21.15.18:53184) 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65240640 1 -- 172.21.15.18:0/2744995163 learned_addr learned my addr 172.21.15.18:0/2744995163 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65a41640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b6006c320 0x7f8b600ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65240640 1 -- 172.21.15.18:0/2744995163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6015e880 msgr2=0x7f8b6006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65240640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6015e880 0x7f8b6006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65240640 1 -- 172.21.15.18:0/2744995163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b6006c320 msgr2=0x7f8b600ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65240640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b6006c320 0x7f8b600ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65240640 1 -- 172.21.15.18:0/2744995163 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b600cac80 con 0x7f8b600b60a0 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65a41640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b6006c320 0x7f8b600ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b66242640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6015e880 0x7f8b6006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.237+0000 7f8b65240640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b600b60a0 0x7f8b6014de30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009890 tx=0x7f8b500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.238+0000 7f8b567fc640 1 -- 172.21.15.18:0/2744995163 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50037040 con 0x7f8b600b60a0 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.238+0000 7f8b567fc640 1 -- 172.21.15.18:0/2744995163 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b50042070 con 0x7f8b600b60a0 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.238+0000 7f8b67ca5640 1 -- 172.21.15.18:0/2744995163 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6009b340 con 0x7f8b600b60a0 2024-09-15T23:36:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.238+0000 7f8b567fc640 1 -- 172.21.15.18:0/2744995163 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50030bd0 con 0x7f8b600b60a0 2024-09-15T23:36:01.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.238+0000 7f8b67ca5640 1 -- 172.21.15.18:0/2744995163 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6009b900 con 0x7f8b600b60a0 2024-09-15T23:36:01.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.238+0000 7f8b567fc640 1 -- 172.21.15.18:0/2744995163 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8b50047460 con 0x7f8b600b60a0 2024-09-15T23:36:01.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.238+0000 7f8b567fc640 1 -- 172.21.15.18:0/2744995163 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f8b5003d9c0 con 0x7f8b600b60a0 2024-09-15T23:36:01.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.239+0000 7f8b67ca5640 1 -- 172.21.15.18:0/2744995163 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b60073520 con 0x7f8b600b60a0 2024-09-15T23:36:01.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.241+0000 7f8b567fc640 1 -- 172.21.15.18:0/2744995163 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8b50032e20 con 0x7f8b600b60a0 2024-09-15T23:36:01.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.312+0000 7f8b67ca5640 1 -- 172.21.15.18:0/2744995163 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a88ba428-0477-4d56-8889-552e0b7229ee", "id": 5} v 0) -- 0x7f8b600b39e0 con 0x7f8b600b60a0 2024-09-15T23:36:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.315+0000 7f8b567fc640 1 -- 172.21.15.18:0/2744995163 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a88ba428-0477-4d56-8889-552e0b7229ee", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f8b50030e10 con 0x7f8b600b60a0 2024-09-15T23:36:01.316 INFO:teuthology.orchestra.run.smithi018.stdout:5 2024-09-15T23:36:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.316+0000 7f8b67ca5640 1 -- 172.21.15.18:0/2744995163 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b600b60a0 msgr2=0x7f8b6014de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.316+0000 7f8b67ca5640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b600b60a0 0x7f8b6014de30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009890 tx=0x7f8b500046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.316+0000 7f8b67ca5640 1 -- 172.21.15.18:0/2744995163 shutdown_connections 2024-09-15T23:36:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.316+0000 7f8b67ca5640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b6015e880 0x7f8b6006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.316+0000 7f8b67ca5640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b600b60a0 0x7f8b6014de30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.316+0000 7f8b67ca5640 1 --2- 172.21.15.18:0/2744995163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b6006c320 0x7f8b600ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.316+0000 7f8b67ca5640 1 -- 172.21.15.18:0/2744995163 >> 172.21.15.18:0/2744995163 conn(0x7f8b600b6a90 msgr2=0x7f8b6015b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.316+0000 7f8b67ca5640 1 -- 172.21.15.18:0/2744995163 shutdown_connections 2024-09-15T23:36:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.316+0000 7f8b67ca5640 1 -- 172.21.15.18:0/2744995163 wait complete. 2024-09-15T23:36:01.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new 023695f4-8073-47f3-94e5-fcec4e708144 6 2024-09-15T23:36:01.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.428+0000 7fea561b9640 1 Processor -- start 2024-09-15T23:36:01.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.428+0000 7fea561b9640 1 -- start start 2024-09-15T23:36:01.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.428+0000 7fea561b9640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea5015c230 0x7fea5015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.428+0000 7fea561b9640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea500b41f0 con 0x7fea500b5760 2024-09-15T23:36:01.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.428+0000 7fea561b9640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea500b44d0 con 0x7fea500b3c30 2024-09-15T23:36:01.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.428+0000 7fea561b9640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fea5015c610 con 0x7fea5015c230 2024-09-15T23:36:01.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.429+0000 7fea47fff640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fea500b3c30 0x7fea500b5050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:43576/0 (socket says 172.21.15.18:43576) 2024-09-15T23:36:01.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.429+0000 7fea47fff640 1 -- 172.21.15.18:0/2610664008 learned_addr learned my addr 172.21.15.18:0/2610664008 (peer_addr_for_me v1:172.21.15.18:0/0) 2024-09-15T23:36:01.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.429+0000 7fea4f7fe640 1 --2- 172.21.15.18:0/2610664008 >> v2:172.21.15.7:3300/0 conn(0x7fea5015c230 0x7fea5015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.429+0000 7fea4effd640 1 -- 172.21.15.18:0/2610664008 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 708071641 0 0) 0x7fea500b44d0 con 0x7fea500b3c30 2024-09-15T23:36:01.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.429+0000 7fea4effd640 1 -- 172.21.15.18:0/2610664008 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea3c003520 con 0x7fea500b3c30 2024-09-15T23:36:01.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.429+0000 7fea4f7fe640 1 -- 172.21.15.18:0/2610664008 >> v1:172.21.15.18:6789/0 conn(0x7fea500b3c30 legacy=0x7fea500b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.429+0000 7fea4f7fe640 1 -- 172.21.15.18:0/2610664008 >> v1:172.21.15.7:6789/0 conn(0x7fea500b5760 legacy=0x7fea500aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.429+0000 7fea4effd640 1 -- 172.21.15.18:0/2610664008 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2519156770 0 0) 0x7fea500b41f0 con 0x7fea500b5760 2024-09-15T23:36:01.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.429+0000 7fea4f7fe640 1 -- 172.21.15.18:0/2610664008 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea500b47b0 con 0x7fea5015c230 2024-09-15T23:36:01.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.430+0000 7fea4f7fe640 1 --2- 172.21.15.18:0/2610664008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea5015c230 0x7fea5015eb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fea400097a0 tx=0x7fea4002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e518e05bf403abba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.430+0000 7fea4effd640 1 -- 172.21.15.18:0/2610664008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40037070 con 0x7fea5015c230 2024-09-15T23:36:01.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.430+0000 7fea4effd640 1 -- 172.21.15.18:0/2610664008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea40032440 con 0x7fea5015c230 2024-09-15T23:36:01.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.430+0000 7fea4effd640 1 -- 172.21.15.18:0/2610664008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400327f0 con 0x7fea5015c230 2024-09-15T23:36:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.431+0000 7fea561b9640 1 -- 172.21.15.18:0/2610664008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea5015c230 msgr2=0x7fea5015eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.431+0000 7fea561b9640 1 --2- 172.21.15.18:0/2610664008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea5015c230 0x7fea5015eb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fea400097a0 tx=0x7fea4002a1a0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.431+0000 7fea561b9640 1 -- 172.21.15.18:0/2610664008 shutdown_connections 2024-09-15T23:36:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.431+0000 7fea561b9640 1 --2- 172.21.15.18:0/2610664008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea5015c230 0x7fea5015eb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.431+0000 7fea561b9640 1 -- 172.21.15.18:0/2610664008 >> 172.21.15.18:0/2610664008 conn(0x7fea50073670 msgr2=0x7fea5006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.431+0000 7fea561b9640 1 -- 172.21.15.18:0/2610664008 shutdown_connections 2024-09-15T23:36:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.431+0000 7fea561b9640 1 -- 172.21.15.18:0/2610664008 wait complete. 2024-09-15T23:36:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.432+0000 7fea561b9640 1 Processor -- start 2024-09-15T23:36:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.432+0000 7fea561b9640 1 -- start start 2024-09-15T23:36:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea561b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea500b3c30 0x7fea501f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea561b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500b5760 0x7fea500cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea561b9640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea5015c230 0x7fea5006f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea561b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea50067b00 con 0x7fea500b5760 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea561b9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fea500b4e00 con 0x7fea500b3c30 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea561b9640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fea500af6e0 con 0x7fea5015c230 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea500b3c30 0x7fea501f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea500b3c30 0x7fea501f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53190/0 (socket says 172.21.15.18:53190) 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4ffff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea5015c230 0x7fea5006f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4f7fe640 1 -- 172.21.15.18:0/1328389476 learned_addr learned my addr 172.21.15.18:0/1328389476 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea47fff640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500b5760 0x7fea500cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4f7fe640 1 -- 172.21.15.18:0/1328389476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea5015c230 msgr2=0x7fea5006f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4f7fe640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea5015c230 0x7fea5006f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4f7fe640 1 -- 172.21.15.18:0/1328389476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500b5760 msgr2=0x7fea500cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4f7fe640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500b5760 0x7fea500cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4f7fe640 1 -- 172.21.15.18:0/1328389476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea501f1c30 con 0x7fea500b3c30 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea4ffff640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea5015c230 0x7fea5006f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.433+0000 7fea47fff640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500b5760 0x7fea500cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.434+0000 7fea4f7fe640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea500b3c30 0x7fea501f3f50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fea4002a1e0 tx=0x7fea400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.434+0000 7fea4cff9640 1 -- 172.21.15.18:0/1328389476 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40041070 con 0x7fea500b3c30 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.434+0000 7fea4cff9640 1 -- 172.21.15.18:0/1328389476 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4003c870 con 0x7fea500b3c30 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.434+0000 7fea4cff9640 1 -- 172.21.15.18:0/1328389476 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40037040 con 0x7fea500b3c30 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.434+0000 7fea561b9640 1 -- 172.21.15.18:0/1328389476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea501f47f0 con 0x7fea500b3c30 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.434+0000 7fea561b9640 1 -- 172.21.15.18:0/1328389476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea501f4490 con 0x7fea500b3c30 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.434+0000 7fea4cff9640 1 -- 172.21.15.18:0/1328389476 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fea40044070 con 0x7fea500b3c30 2024-09-15T23:36:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.434+0000 7fea4cff9640 1 -- 172.21.15.18:0/1328389476 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fea40004020 con 0x7fea500b3c30 2024-09-15T23:36:01.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.435+0000 7fea561b9640 1 -- 172.21.15.18:0/1328389476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5015eb10 con 0x7fea500b3c30 2024-09-15T23:36:01.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.437+0000 7fea4cff9640 1 -- 172.21.15.18:0/1328389476 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fea4004b070 con 0x7fea500b3c30 2024-09-15T23:36:01.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.507+0000 7fea561b9640 1 -- 172.21.15.18:0/1328389476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "023695f4-8073-47f3-94e5-fcec4e708144", "id": 6} v 0) -- 0x7fea5015e7d0 con 0x7fea500b3c30 2024-09-15T23:36:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.510+0000 7fea4cff9640 1 -- 172.21.15.18:0/1328389476 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "023695f4-8073-47f3-94e5-fcec4e708144", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fea4003cc60 con 0x7fea500b3c30 2024-09-15T23:36:01.511 INFO:teuthology.orchestra.run.smithi018.stdout:6 2024-09-15T23:36:01.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.511+0000 7fea561b9640 1 -- 172.21.15.18:0/1328389476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea500b3c30 msgr2=0x7fea501f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.511+0000 7fea561b9640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea500b3c30 0x7fea501f3f50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fea4002a1e0 tx=0x7fea400046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.511+0000 7fea561b9640 1 -- 172.21.15.18:0/1328389476 shutdown_connections 2024-09-15T23:36:01.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.511+0000 7fea561b9640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea5015c230 0x7fea5006f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.511+0000 7fea561b9640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500b5760 0x7fea500cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.511+0000 7fea561b9640 1 --2- 172.21.15.18:0/1328389476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea500b3c30 0x7fea501f3f50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.511+0000 7fea561b9640 1 -- 172.21.15.18:0/1328389476 >> 172.21.15.18:0/1328389476 conn(0x7fea50073670 msgr2=0x7fea500b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.511+0000 7fea561b9640 1 -- 172.21.15.18:0/1328389476 shutdown_connections 2024-09-15T23:36:01.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.511+0000 7fea561b9640 1 -- 172.21.15.18:0/1328389476 wait complete. 2024-09-15T23:36:01.523 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new 7f04fe2f-99cf-4cde-a003-d38d453334dc 7 2024-09-15T23:36:01.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.623+0000 7f0e1e911640 1 Processor -- start 2024-09-15T23:36:01.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.623+0000 7f0e1e911640 1 -- start start 2024-09-15T23:36:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.624+0000 7f0e1e911640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0e1815c230 0x7f0e1815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.624+0000 7f0e1e911640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e180b5760 0x7f0e180aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.624+0000 7f0e1e911640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e180b41f0 con 0x7f0e180b3c30 2024-09-15T23:36:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.624+0000 7f0e1e911640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1815c610 con 0x7f0e1815c230 2024-09-15T23:36:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.624+0000 7f0e1e911640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f0e180b4e00 con 0x7f0e180b5760 2024-09-15T23:36:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.624+0000 7f0e1ceae640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e180b5760 0x7f0e180aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.624+0000 7f0e17fff640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0e1815c230 0x7f0e1815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.624+0000 7f0e1ceae640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e180b5760 0x7f0e180aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50502/0 (socket says 172.21.15.18:50502) 2024-09-15T23:36:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.624+0000 7f0e1ceae640 1 -- 172.21.15.18:0/4044887807 learned_addr learned my addr 172.21.15.18:0/4044887807 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:01.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.625+0000 7f0e17fff640 1 -- 172.21.15.18:0/4044887807 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e180b5760 msgr2=0x7f0e180aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:01.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.625+0000 7f0e17fff640 1 --2- 172.21.15.18:0/4044887807 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e180b5760 0x7f0e180aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.625+0000 7f0e17fff640 1 -- 172.21.15.18:0/4044887807 >> v1:172.21.15.18:6789/0 conn(0x7f0e180b3c30 legacy=0x7f0e180b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.625+0000 7f0e17fff640 1 -- 172.21.15.18:0/4044887807 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e180b44d0 con 0x7f0e1815c230 2024-09-15T23:36:01.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.625+0000 7f0e1ceae640 1 --2- 172.21.15.18:0/4044887807 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e180b5760 0x7f0e180aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:01.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.625+0000 7f0e17fff640 1 --2- 172.21.15.18:0/4044887807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1815c230 0x7f0e1815eb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e080097a0 tx=0x7f0e0802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d907a913a7bf8ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e16ffd640 1 -- 172.21.15.18:0/4044887807 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08037070 con 0x7f0e1815c230 2024-09-15T23:36:01.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e16ffd640 1 -- 172.21.15.18:0/4044887807 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0802ae50 con 0x7f0e1815c230 2024-09-15T23:36:01.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e16ffd640 1 -- 172.21.15.18:0/4044887807 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08032660 con 0x7f0e1815c230 2024-09-15T23:36:01.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e1e911640 1 -- 172.21.15.18:0/4044887807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1815c230 msgr2=0x7f0e1815eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e1e911640 1 --2- 172.21.15.18:0/4044887807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1815c230 0x7f0e1815eb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e080097a0 tx=0x7f0e0802a100 comp rx=0 tx=0).stop 2024-09-15T23:36:01.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e1e911640 1 -- 172.21.15.18:0/4044887807 shutdown_connections 2024-09-15T23:36:01.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e1e911640 1 --2- 172.21.15.18:0/4044887807 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e180b5760 0x7f0e180aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e1e911640 1 --2- 172.21.15.18:0/4044887807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1815c230 0x7f0e1815eb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e1e911640 1 -- 172.21.15.18:0/4044887807 >> 172.21.15.18:0/4044887807 conn(0x7f0e18073670 msgr2=0x7f0e1806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e1e911640 1 -- 172.21.15.18:0/4044887807 shutdown_connections 2024-09-15T23:36:01.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.626+0000 7f0e1e911640 1 -- 172.21.15.18:0/4044887807 wait complete. 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.628+0000 7f0e1e911640 1 Processor -- start 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.628+0000 7f0e1e911640 1 -- start start 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1e911640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e180b3c30 0x7f0e1814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1e911640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e180b5760 0x7f0e180b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1e911640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e1815c230 0x7f0e18150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1e911640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18067b00 con 0x7f0e180b5760 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1e911640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0e180b4a90 con 0x7f0e180b3c30 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1e911640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f0e180af510 con 0x7f0e1815c230 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e17fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e180b3c30 0x7f0e1814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e17fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e180b3c30 0x7f0e1814e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53204/0 (socket says 172.21.15.18:53204) 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e17fff640 1 -- 172.21.15.18:0/2972504809 learned_addr learned my addr 172.21.15.18:0/2972504809 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1ceae640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e1815c230 0x7f0e18150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e177fe640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e180b5760 0x7f0e180b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e17fff640 1 -- 172.21.15.18:0/2972504809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e1815c230 msgr2=0x7f0e18150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e17fff640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e1815c230 0x7f0e18150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e17fff640 1 -- 172.21.15.18:0/2972504809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e180b5760 msgr2=0x7f0e180b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e17fff640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e180b5760 0x7f0e180b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e17fff640 1 -- 172.21.15.18:0/2972504809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e181e7f40 con 0x7f0e180b3c30 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1ceae640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e1815c230 0x7f0e18150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e17fff640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e180b3c30 0x7f0e1814e950 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e08035040 tx=0x7f0e080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e14ff9640 1 -- 172.21.15.18:0/2972504809 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08041070 con 0x7f0e180b3c30 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e14ff9640 1 -- 172.21.15.18:0/2972504809 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0803c460 con 0x7f0e180b3c30 2024-09-15T23:36:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1e911640 1 -- 172.21.15.18:0/2972504809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e181e6ea0 con 0x7f0e180b3c30 2024-09-15T23:36:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e14ff9640 1 -- 172.21.15.18:0/2972504809 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08037040 con 0x7f0e180b3c30 2024-09-15T23:36:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.629+0000 7f0e1e911640 1 -- 172.21.15.18:0/2972504809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e181e7460 con 0x7f0e180b3c30 2024-09-15T23:36:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.630+0000 7f0e14ff9640 1 -- 172.21.15.18:0/2972504809 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e0803a070 con 0x7f0e180b3c30 2024-09-15T23:36:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.630+0000 7f0e14ff9640 1 -- 172.21.15.18:0/2972504809 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f0e08045d80 con 0x7f0e180b3c30 2024-09-15T23:36:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.631+0000 7f0e1e911640 1 -- 172.21.15.18:0/2972504809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e180aefd0 con 0x7f0e180b3c30 2024-09-15T23:36:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.633+0000 7f0e14ff9640 1 -- 172.21.15.18:0/2972504809 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0e08045930 con 0x7f0e180b3c30 2024-09-15T23:36:01.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.703+0000 7f0e1e911640 1 -- 172.21.15.18:0/2972504809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7f04fe2f-99cf-4cde-a003-d38d453334dc", "id": 7} v 0) -- 0x7f0e18074210 con 0x7f0e180b3c30 2024-09-15T23:36:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.707+0000 7f0e14ff9640 1 -- 172.21.15.18:0/2972504809 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7f04fe2f-99cf-4cde-a003-d38d453334dc", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f0e08045b10 con 0x7f0e180b3c30 2024-09-15T23:36:01.708 INFO:teuthology.orchestra.run.smithi018.stdout:7 2024-09-15T23:36:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.707+0000 7f0e1e911640 1 -- 172.21.15.18:0/2972504809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e180b3c30 msgr2=0x7f0e1814e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.707+0000 7f0e1e911640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e180b3c30 0x7f0e1814e950 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e08035040 tx=0x7f0e080046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.707+0000 7f0e1e911640 1 -- 172.21.15.18:0/2972504809 shutdown_connections 2024-09-15T23:36:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.707+0000 7f0e1e911640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e1815c230 0x7f0e18150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.707+0000 7f0e1e911640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e180b5760 0x7f0e180b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.707+0000 7f0e1e911640 1 --2- 172.21.15.18:0/2972504809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e180b3c30 0x7f0e1814e950 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.707+0000 7f0e1e911640 1 -- 172.21.15.18:0/2972504809 >> 172.21.15.18:0/2972504809 conn(0x7f0e18073670 msgr2=0x7f0e180b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.708+0000 7f0e1e911640 1 -- 172.21.15.18:0/2972504809 shutdown_connections 2024-09-15T23:36:01.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:01.708+0000 7f0e1e911640 1 -- 172.21.15.18:0/2972504809 wait complete. 2024-09-15T23:36:01.719 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T23:36:01.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T23:36:01.722 INFO:tasks.ceph.osd.0:Started 2024-09-15T23:36:01.722 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T23:36:01.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T23:36:01.724 INFO:tasks.ceph.osd.1:Started 2024-09-15T23:36:01.724 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T23:36:01.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T23:36:01.726 INFO:tasks.ceph.osd.2:Started 2024-09-15T23:36:01.726 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T23:36:01.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T23:36:01.728 INFO:tasks.ceph.osd.3:Started 2024-09-15T23:36:01.729 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T23:36:01.729 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T23:36:01.731 INFO:tasks.ceph.osd.4:Started 2024-09-15T23:36:01.731 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T23:36:01.731 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T23:36:01.733 INFO:tasks.ceph.osd.5:Started 2024-09-15T23:36:01.733 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-15T23:36:01.733 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-15T23:36:01.735 INFO:tasks.ceph.osd.6:Started 2024-09-15T23:36:01.735 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-15T23:36:01.735 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-15T23:36:01.737 INFO:tasks.ceph.osd.7:Started 2024-09-15T23:36:01.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:01.779 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:01.778+0000 7f57c4c0e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:36:01.852 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:01.851+0000 7f57c4c0e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:36:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.875+0000 7fd0288d9640 1 Processor -- start 2024-09-15T23:36:01.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.875+0000 7fd0288d9640 1 -- start start 2024-09-15T23:36:01.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd0288d9640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd02415e800 0x7fd0240b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd0288d9640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0240ae250 0x7fd0240b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd0288d9640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 0x7fd0240b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd0288d9640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd0240b6030 con 0x7fd0240ae250 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd0288d9640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fd0240b67e0 con 0x7fd02415e800 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd0288d9640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fd0240b1ac0 con 0x7fd0240b29d0 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd021d9b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0240ae250 0x7fd0240b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd021d9b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0240ae250 0x7fd0240b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42856/0 (socket says 172.21.15.7:42856) 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd021d9b640 1 -- 172.21.15.7:0/933852560 learned_addr learned my addr 172.21.15.7:0/933852560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd022d9d640 1 --2- 172.21.15.7:0/933852560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 0x7fd0240b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd02259c640 1 --2- 172.21.15.7:0/933852560 >> v2:172.21.15.18:3300/0 conn(0x7fd02415e800 0x7fd0240b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd021d9b640 1 -- 172.21.15.7:0/933852560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 msgr2=0x7fd0240b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd021d9b640 1 --2- 172.21.15.7:0/933852560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 0x7fd0240b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd021d9b640 1 -- 172.21.15.7:0/933852560 >> v2:172.21.15.18:3300/0 conn(0x7fd02415e800 msgr2=0x7fd0240b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd021d9b640 1 --2- 172.21.15.7:0/933852560 >> v2:172.21.15.18:3300/0 conn(0x7fd02415e800 0x7fd0240b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.876+0000 7fd021d9b640 1 -- 172.21.15.7:0/933852560 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02409ae70 con 0x7fd0240ae250 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.877+0000 7fd021d9b640 1 --2- 172.21.15.7:0/933852560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0240ae250 0x7fd0240b2490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd024073970 tx=0x7fd01802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae92940d782474a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.877+0000 7fd02159a640 1 -- 172.21.15.7:0/933852560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018037070 con 0x7fd0240ae250 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.877+0000 7fd02159a640 1 -- 172.21.15.7:0/933852560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd018032440 con 0x7fd0240ae250 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.877+0000 7fd02159a640 1 -- 172.21.15.7:0/933852560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018032810 con 0x7fd0240ae250 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.878+0000 7fd0288d9640 1 -- 172.21.15.7:0/933852560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0240ae250 msgr2=0x7fd0240b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.878+0000 7fd0288d9640 1 --2- 172.21.15.7:0/933852560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0240ae250 0x7fd0240b2490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd024073970 tx=0x7fd01802a1e0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.878+0000 7fd0288d9640 1 -- 172.21.15.7:0/933852560 shutdown_connections 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.878+0000 7fd0288d9640 1 --2- 172.21.15.7:0/933852560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 0x7fd0240b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.878+0000 7fd0288d9640 1 --2- 172.21.15.7:0/933852560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0240ae250 0x7fd0240b2490 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.878+0000 7fd0288d9640 1 --2- 172.21.15.7:0/933852560 >> v2:172.21.15.18:3300/0 conn(0x7fd02415e800 0x7fd0240b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.878+0000 7fd0288d9640 1 -- 172.21.15.7:0/933852560 >> 172.21.15.7:0/933852560 conn(0x7fd024073490 msgr2=0x7fd0240b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.878+0000 7fd0288d9640 1 -- 172.21.15.7:0/933852560 shutdown_connections 2024-09-15T23:36:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.878+0000 7fd0288d9640 1 -- 172.21.15.7:0/933852560 wait complete. 2024-09-15T23:36:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.880+0000 7fd0288d9640 1 Processor -- start 2024-09-15T23:36:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.880+0000 7fd0288d9640 1 -- start start 2024-09-15T23:36:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.880+0000 7fd0288d9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0240ae250 0x7fd0241604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.880+0000 7fd0288d9640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 0x7fd0241537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd0288d9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd02415e800 0x7fd0241571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd0288d9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0240b0560 con 0x7fd02415e800 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd0288d9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd0240b6260 con 0x7fd0240ae250 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd0288d9640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fd0240b06e0 con 0x7fd0240b29d0 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd02415e800 0x7fd0241571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd021d9b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 0x7fd0241537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd02415e800 0x7fd0241571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42872/0 (socket says 172.21.15.7:42872) 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd022d9d640 1 -- 172.21.15.7:0/3596037332 learned_addr learned my addr 172.21.15.7:0/3596037332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd02259c640 1 --2- 172.21.15.7:0/3596037332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0240ae250 0x7fd0241604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd022d9d640 1 -- 172.21.15.7:0/3596037332 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 msgr2=0x7fd0241537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd022d9d640 1 --2- 172.21.15.7:0/3596037332 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 0x7fd0241537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd022d9d640 1 -- 172.21.15.7:0/3596037332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0240ae250 msgr2=0x7fd0241604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd022d9d640 1 --2- 172.21.15.7:0/3596037332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0240ae250 0x7fd0241604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd022d9d640 1 -- 172.21.15.7:0/3596037332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd024152bb0 con 0x7fd02415e800 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd022d9d640 1 --2- 172.21.15.7:0/3596037332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd02415e800 0x7fd0241571e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00ef00 tx=0x7fd01c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd012ffd640 1 -- 172.21.15.7:0/3596037332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c018020 con 0x7fd02415e800 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd0288d9640 1 -- 172.21.15.7:0/3596037332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd024152640 con 0x7fd02415e800 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd012ffd640 1 -- 172.21.15.7:0/3596037332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd01c014400 con 0x7fd02415e800 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd012ffd640 1 -- 172.21.15.7:0/3596037332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0147f0 con 0x7fd02415e800 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.881+0000 7fd0288d9640 1 -- 172.21.15.7:0/3596037332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02415f700 con 0x7fd02415e800 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.882+0000 7fd012ffd640 1 -- 172.21.15.7:0/3596037332 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd01c005d10 con 0x7fd02415e800 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.882+0000 7fd012ffd640 1 -- 172.21.15.7:0/3596037332 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd01c01d400 con 0x7fd02415e800 2024-09-15T23:36:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.883+0000 7fd0288d9640 1 -- 172.21.15.7:0/3596037332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02415ec70 con 0x7fd02415e800 2024-09-15T23:36:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.885+0000 7fd012ffd640 1 -- 172.21.15.7:0/3596037332 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd01c022070 con 0x7fd02415e800 2024-09-15T23:36:01.889 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:01.888+0000 7fe506e1c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:36:01.945 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:01.944+0000 7fe506e1c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:36:01.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.945+0000 7fd0288d9640 1 -- 172.21.15.7:0/3596037332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0240af8d0 con 0x7fd02415e800 2024-09-15T23:36:01.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.945+0000 7fd012ffd640 1 -- 172.21.15.7:0/3596037332 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd024152bb0 con 0x7fd02415e800 2024-09-15T23:36:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:01.946 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:01.705603+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:01.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.946+0000 7fd0288d9640 1 -- 172.21.15.7:0/3596037332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd02415e800 msgr2=0x7fd0241571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:01.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.946+0000 7fd0288d9640 1 --2- 172.21.15.7:0/3596037332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd02415e800 0x7fd0241571e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00ef00 tx=0x7fd01c00b480 comp rx=0 tx=0).stop 2024-09-15T23:36:01.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.946+0000 7fd0288d9640 1 -- 172.21.15.7:0/3596037332 shutdown_connections 2024-09-15T23:36:01.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.946+0000 7fd0288d9640 1 --2- 172.21.15.7:0/3596037332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd02415e800 0x7fd0241571e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.946+0000 7fd0288d9640 1 --2- 172.21.15.7:0/3596037332 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0240b29d0 0x7fd0241537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.946+0000 7fd0288d9640 1 --2- 172.21.15.7:0/3596037332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0240ae250 0x7fd0241604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:01.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.946+0000 7fd0288d9640 1 -- 172.21.15.7:0/3596037332 >> 172.21.15.7:0/3596037332 conn(0x7fd024073490 msgr2=0x7fd0240ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:01.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.946+0000 7fd0288d9640 1 -- 172.21.15.7:0/3596037332 shutdown_connections 2024-09-15T23:36:01.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:01.946+0000 7fd0288d9640 1 -- 172.21.15.7:0/3596037332 wait complete. 2024-09-15T23:36:01.961 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T23:36:01.961 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T23:36:02.247 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.246+0000 7f57c4c0e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:36:02.331 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:02.330+0000 7fe506e1c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:36:02.336 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.335+0000 7f57c4c0e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:36:02.420 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:02.419+0000 7fe506e1c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:36:02.448 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.448+0000 7f57c4c0e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:36:02.497 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.496+0000 7f57c4c0e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:36:02.531 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:02.530+0000 7fe506e1c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:36:02.579 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:02.579+0000 7fe506e1c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:36:02.615 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.615+0000 7f57c4c0e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:36:02.671 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.671+0000 7f57c4c0e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:36:02.693 INFO:tasks.ceph.osd.4.smithi018.stderr:2024-09-15T23:36:02.692+0000 7f07976e0640 -1 Falling back to public interface 2024-09-15T23:36:02.698 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:02.697+0000 7fe506e1c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:36:02.716 INFO:tasks.ceph.osd.5.smithi018.stderr:2024-09-15T23:36:02.715+0000 7ff7f8813640 -1 Falling back to public interface 2024-09-15T23:36:02.719 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.718+0000 7f57c4c0e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:36:02.754 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:02.753+0000 7fe506e1c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:36:02.767 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:02.801 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:02.800+0000 7fe506e1c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:36:02.837 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.836+0000 7f57c4c0e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:36:02.861 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.860+0000 7efd82bd8640 1 Processor -- start 2024-09-15T23:36:02.861 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.860+0000 7efd82bd8640 1 -- start start 2024-09-15T23:36:02.862 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd82bd8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 0x7efd7c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:02.862 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd82bd8640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efd7c0b09d0 0x7efd7c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:02.862 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd82bd8640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd7c0b5ff0 con 0x7efd7c0701a0 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd82bd8640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7efd7c0b6600 con 0x7efd7c0b09d0 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd82bd8640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c0b45c0 con 0x7efd7c06f3a0 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd80974640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efd7c0701a0 0x7efd7c0b02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52292/0 (socket says 172.21.15.7:52292) 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd80974640 1 -- 172.21.15.7:0/1400879175 learned_addr learned my addr 172.21.15.7:0/1400879175 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd81175640 1 --2- 172.21.15.7:0/1400879175 >> v2:172.21.15.18:3300/0 conn(0x7efd7c0b09d0 0x7efd7c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd7bfff640 1 --2- 172.21.15.7:0/1400879175 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 0x7efd7c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd7b7fe640 1 -- 172.21.15.7:0/1400879175 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2128966456 0 0) 0x7efd7c0b5ff0 con 0x7efd7c0701a0 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.861+0000 7efd7b7fe640 1 -- 172.21.15.7:0/1400879175 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd6c003520 con 0x7efd7c0701a0 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.862+0000 7efd81175640 1 -- 172.21.15.7:0/1400879175 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 msgr2=0x7efd7c06c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.862+0000 7efd81175640 1 --2- 172.21.15.7:0/1400879175 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 0x7efd7c06c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:02.863 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.862+0000 7efd81175640 1 -- 172.21.15.7:0/1400879175 >> v1:172.21.15.7:6789/0 conn(0x7efd7c0701a0 legacy=0x7efd7c0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:02.864 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.862+0000 7efd81175640 1 -- 172.21.15.7:0/1400879175 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c0b48a0 con 0x7efd7c0b09d0 2024-09-15T23:36:02.865 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.862+0000 7efd81175640 1 --2- 172.21.15.7:0/1400879175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd7c0b09d0 0x7efd7c15ed90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd74009d90 tx=0x7efd7402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fc4b57db7bd0732 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:02.865 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd7b7fe640 1 -- 172.21.15.7:0/1400879175 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74007740 con 0x7efd7c0b09d0 2024-09-15T23:36:02.865 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd7b7fe640 1 -- 172.21.15.7:0/1400879175 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd740078e0 con 0x7efd7c0b09d0 2024-09-15T23:36:02.865 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd7b7fe640 1 -- 172.21.15.7:0/1400879175 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74007c90 con 0x7efd7c0b09d0 2024-09-15T23:36:02.865 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd82bd8640 1 -- 172.21.15.7:0/1400879175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd7c0b09d0 msgr2=0x7efd7c15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:02.865 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd82bd8640 1 --2- 172.21.15.7:0/1400879175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd7c0b09d0 0x7efd7c15ed90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd74009d90 tx=0x7efd7402a660 comp rx=0 tx=0).stop 2024-09-15T23:36:02.866 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd82bd8640 1 -- 172.21.15.7:0/1400879175 shutdown_connections 2024-09-15T23:36:02.866 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd82bd8640 1 --2- 172.21.15.7:0/1400879175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd7c0b09d0 0x7efd7c15ed90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:02.866 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd82bd8640 1 --2- 172.21.15.7:0/1400879175 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 0x7efd7c06c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:02.866 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd82bd8640 1 -- 172.21.15.7:0/1400879175 >> 172.21.15.7:0/1400879175 conn(0x7efd7c0b6a40 msgr2=0x7efd7c0b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:02.866 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd82bd8640 1 -- 172.21.15.7:0/1400879175 shutdown_connections 2024-09-15T23:36:02.866 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.863+0000 7efd82bd8640 1 -- 172.21.15.7:0/1400879175 wait complete. 2024-09-15T23:36:02.866 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.864+0000 7efd82bd8640 1 Processor -- start 2024-09-15T23:36:02.866 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.864+0000 7efd82bd8640 1 -- start start 2024-09-15T23:36:02.868 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd82bd8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 0x7efd7c09c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:02.868 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd82bd8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd7c0701a0 0x7efd7c1e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd82bd8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd7c0b09d0 0x7efd7c15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd82bd8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c070e70 con 0x7efd7c0b09d0 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd82bd8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c06da40 con 0x7efd7c0701a0 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd82bd8640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c070ff0 con 0x7efd7c06f3a0 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd81175640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd7c0b09d0 0x7efd7c15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd81175640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd7c0b09d0 0x7efd7c15a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42880/0 (socket says 172.21.15.7:42880) 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd81175640 1 -- 172.21.15.7:0/3444908931 learned_addr learned my addr 172.21.15.7:0/3444908931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd7bfff640 1 --2- 172.21.15.7:0/3444908931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd7c0701a0 0x7efd7c1e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.865+0000 7efd80974640 1 --2- 172.21.15.7:0/3444908931 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 0x7efd7c09c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd81175640 1 -- 172.21.15.7:0/3444908931 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 msgr2=0x7efd7c09c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd81175640 1 --2- 172.21.15.7:0/3444908931 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 0x7efd7c09c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd81175640 1 -- 172.21.15.7:0/3444908931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd7c0701a0 msgr2=0x7efd7c1e7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd81175640 1 --2- 172.21.15.7:0/3444908931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd7c0701a0 0x7efd7c1e7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:02.869 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd81175640 1 -- 172.21.15.7:0/3444908931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c09c850 con 0x7efd7c0b09d0 2024-09-15T23:36:02.870 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd81175640 1 --2- 172.21.15.7:0/3444908931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd7c0b09d0 0x7efd7c15a6a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efd74009eb0 tx=0x7efd74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:02.870 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd797fa640 1 -- 172.21.15.7:0/3444908931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7403f070 con 0x7efd7c0b09d0 2024-09-15T23:36:02.870 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd797fa640 1 -- 172.21.15.7:0/3444908931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd74031c60 con 0x7efd7c0b09d0 2024-09-15T23:36:02.870 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd797fa640 1 -- 172.21.15.7:0/3444908931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74009450 con 0x7efd7c0b09d0 2024-09-15T23:36:02.870 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd82bd8640 1 -- 172.21.15.7:0/3444908931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c09cb30 con 0x7efd7c0b09d0 2024-09-15T23:36:02.870 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd82bd8640 1 -- 172.21.15.7:0/3444908931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c09dae0 con 0x7efd7c0b09d0 2024-09-15T23:36:02.870 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd797fa640 1 -- 172.21.15.7:0/3444908931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd740439e0 con 0x7efd7c0b09d0 2024-09-15T23:36:02.870 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.866+0000 7efd797fa640 1 -- 172.21.15.7:0/3444908931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7efd7403be00 con 0x7efd7c0b09d0 2024-09-15T23:36:02.870 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.868+0000 7efd82bd8640 1 -- 172.21.15.7:0/3444908931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c06c5a0 con 0x7efd7c0b09d0 2024-09-15T23:36:02.871 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.870+0000 7efd797fa640 1 -- 172.21.15.7:0/3444908931 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7efd7403b730 con 0x7efd7c0b09d0 2024-09-15T23:36:02.892 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.891+0000 7f57c4c0e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:36:02.918 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:02.918+0000 7fe506e1c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:36:02.931 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.930+0000 7efd82bd8640 1 -- 172.21.15.7:0/3444908931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd7c06f850 con 0x7efd7c0b09d0 2024-09-15T23:36:02.931 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.930+0000 7efd797fa640 1 -- 172.21.15.7:0/3444908931 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7efd7403b910 con 0x7efd7c0b09d0 2024-09-15T23:36:02.931 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-15T23:36:02.932 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":10,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:01.705603+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:02.933 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.931+0000 7efd82bd8640 1 -- 172.21.15.7:0/3444908931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd7c0b09d0 msgr2=0x7efd7c15a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:02.933 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.931+0000 7efd82bd8640 1 --2- 172.21.15.7:0/3444908931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd7c0b09d0 0x7efd7c15a6a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efd74009eb0 tx=0x7efd74004a40 comp rx=0 tx=0).stop 2024-09-15T23:36:02.933 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.931+0000 7efd82bd8640 1 -- 172.21.15.7:0/3444908931 shutdown_connections 2024-09-15T23:36:02.933 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.931+0000 7efd82bd8640 1 --2- 172.21.15.7:0/3444908931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd7c0b09d0 0x7efd7c15a6a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:02.933 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.931+0000 7efd82bd8640 1 --2- 172.21.15.7:0/3444908931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd7c0701a0 0x7efd7c1e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:02.933 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.931+0000 7efd82bd8640 1 --2- 172.21.15.7:0/3444908931 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd7c06f3a0 0x7efd7c09c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:02.933 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.931+0000 7efd82bd8640 1 -- 172.21.15.7:0/3444908931 >> 172.21.15.7:0/3444908931 conn(0x7efd7c0b6a40 msgr2=0x7efd7c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:02.933 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.931+0000 7efd82bd8640 1 -- 172.21.15.7:0/3444908931 shutdown_connections 2024-09-15T23:36:02.933 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:02.931+0000 7efd82bd8640 1 -- 172.21.15.7:0/3444908931 wait complete. 2024-09-15T23:36:02.941 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-15T23:36:02.947 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:02.946+0000 7f57c4c0e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:36:02.971 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:02.970+0000 7fe506e1c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:36:03.019 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:03.018+0000 7f57c4c0e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:36:03.024 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:03.023+0000 7fe506e1c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:36:03.065 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:03.065+0000 7f57c4c0e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:36:03.095 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:03.094+0000 7fe506e1c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:36:03.112 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:03.111+0000 7f57c4c0e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:36:03.141 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:03.140+0000 7fe506e1c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:36:03.187 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:03.186+0000 7fe506e1c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:36:03.208 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-15T23:36:03.207+0000 7f397c6b0640 -1 Falling back to public interface 2024-09-15T23:36:03.208 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-15T23:36:03.207+0000 7f6d2ca0c640 -1 Falling back to public interface 2024-09-15T23:36:03.209 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-15T23:36:03.207+0000 7f397b658640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2024-09-15T23:36:03.435 INFO:tasks.ceph.osd.7.smithi018.stderr:2024-09-15T23:36:03.434+0000 7fed9cc1d640 -1 Falling back to public interface 2024-09-15T23:36:03.473 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-15T23:36:03.473+0000 7f11deba8640 -1 Falling back to public interface 2024-09-15T23:36:03.474 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-15T23:36:03.473+0000 7fa6ee40f640 -1 Falling back to public interface 2024-09-15T23:36:03.475 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-15T23:36:03.474+0000 7fa6ecc9f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6818/0: (98) Address already in use 2024-09-15T23:36:03.705 INFO:tasks.ceph.osd.6.smithi018.stderr:2024-09-15T23:36:03.704+0000 7f6f9aa1a640 -1 Falling back to public interface 2024-09-15T23:36:03.802 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:03.802+0000 7f57c4c0e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:36:03.878 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:36:03.877+0000 7fe506e1c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:36:04.818 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:04.817+0000 7f57648f6640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T23:36:04.818 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:36:04.817+0000 7f57648f6640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T23:36:06.480 INFO:tasks.ceph.osd.4.smithi018.stderr:2024-09-15T23:36:06.479+0000 7f07976e0640 -1 osd.4 0 log_to_monitors true 2024-09-15T23:36:06.783 INFO:tasks.ceph.osd.5.smithi018.stderr:2024-09-15T23:36:06.783+0000 7ff7f8813640 -1 osd.5 0 log_to_monitors true 2024-09-15T23:36:07.072 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-15T23:36:07.071+0000 7f397c6b0640 -1 osd.2 0 log_to_monitors true 2024-09-15T23:36:07.316 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-15T23:36:07.315+0000 7f11deba8640 -1 osd.1 0 log_to_monitors true 2024-09-15T23:36:07.317 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-15T23:36:07.316+0000 7f6d2ca0c640 -1 osd.3 0 log_to_monitors true 2024-09-15T23:36:07.325 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-15T23:36:07.324+0000 7fa6ee40f640 -1 osd.0 0 log_to_monitors true 2024-09-15T23:36:07.447 INFO:tasks.ceph.osd.7.smithi018.stderr:2024-09-15T23:36:07.446+0000 7fed9cc1d640 -1 osd.7 0 log_to_monitors true 2024-09-15T23:36:07.496 INFO:tasks.ceph.osd.6.smithi018.stderr:2024-09-15T23:36:07.495+0000 7f6f9aa1a640 -1 osd.6 0 log_to_monitors true 2024-09-15T23:36:08.705 INFO:tasks.ceph.osd.4.smithi018.stderr:2024-09-15T23:36:08.703+0000 7f0793e83640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T23:36:08.714 INFO:tasks.ceph.osd.5.smithi018.stderr:2024-09-15T23:36:08.712+0000 7ff7f491a640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T23:36:08.720 INFO:tasks.ceph.osd.4.smithi018.stderr:2024-09-15T23:36:08.718+0000 7f078b48f640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:36:08.723 INFO:tasks.ceph.osd.5.smithi018.stderr:2024-09-15T23:36:08.722+0000 7ff7ebf26640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:36:09.725 INFO:tasks.ceph.osd.7.smithi018.stderr:2024-09-15T23:36:09.723+0000 7fed985e8640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T23:36:09.726 INFO:tasks.ceph.osd.6.smithi018.stderr:2024-09-15T23:36:09.724+0000 7f6f963e8640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T23:36:09.740 INFO:tasks.ceph.osd.6.smithi018.stderr:2024-09-15T23:36:09.738+0000 7f6f8e208640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:36:09.740 INFO:tasks.ceph.osd.7.smithi018.stderr:2024-09-15T23:36:09.738+0000 7fed90408640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:36:09.747 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:09.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.907+0000 7fba8f7b7640 1 Processor -- start 2024-09-15T23:36:09.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.908+0000 7fba8f7b7640 1 -- start start 2024-09-15T23:36:09.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.909+0000 7fba8f7b7640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fba881669e0 0x7fba880d5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:09.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.909+0000 7fba8d553640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fba881669e0 0x7fba880d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:09.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.909+0000 7fba8cd52640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fba880ced90 0x7fba88166dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:34518/0 (socket says 172.21.15.7:34518) 2024-09-15T23:36:09.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.910+0000 7fba8d553640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fba881669e0 0x7fba880d5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:38494/0 (socket says 172.21.15.7:38494) 2024-09-15T23:36:09.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.910+0000 7fba8dd54640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fba881674d0 0x7fba880d3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42008/0 (socket says 172.21.15.7:42008) 2024-09-15T23:36:09.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.910+0000 7fba8cd52640 1 -- 172.21.15.7:0/2962287126 learned_addr learned my addr 172.21.15.7:0/2962287126 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:09.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.910+0000 7fba8f7b7640 1 -- 172.21.15.7:0/2962287126 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba880b7730 con 0x7fba881674d0 2024-09-15T23:36:09.913 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.910+0000 7fba8f7b7640 1 -- 172.21.15.7:0/2962287126 --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba880ba120 con 0x7fba880ced90 2024-09-15T23:36:09.913 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.910+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831666726 0 0) 0x7fba880b7730 con 0x7fba881674d0 2024-09-15T23:36:09.913 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.911+0000 7fba8f7b7640 1 -- 172.21.15.7:0/2962287126 --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fba880b9ef0 con 0x7fba881669e0 2024-09-15T23:36:09.913 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.911+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba74003520 con 0x7fba881674d0 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.911+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2610477403 0 0) 0x7fba880ba120 con 0x7fba880ced90 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.911+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba880b7730 con 0x7fba880ced90 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.911+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3845337806 0 0) 0x7fba74003520 con 0x7fba881674d0 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.911+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 >> v2:172.21.15.18:3300/0 conn(0x7fba881669e0 msgr2=0x7fba880d5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.911+0000 7fba7ffff640 1 --2- 172.21.15.7:0/2962287126 >> v2:172.21.15.18:3300/0 conn(0x7fba881669e0 0x7fba880d5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.911+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 >> v1:172.21.15.18:6789/0 conn(0x7fba880ced90 legacy=0x7fba88166dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.911+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba880ba400 con 0x7fba881674d0 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7fba800019f0 con 0x7fba881674d0 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 >> v1:172.21.15.7:6789/0 conn(0x7fba881674d0 legacy=0x7fba880d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba7ffff640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba74004ba0 0x7fba74004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba7ffff640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba740037a0 0x7fba74003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba7ffff640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba74004160 0x7fba7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:09.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba7400c0c0 con 0x7fba74004160 2024-09-15T23:36:09.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba7400c270 con 0x7fba74004ba0 2024-09-15T23:36:09.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fba7400c470 con 0x7fba740037a0 2024-09-15T23:36:09.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba8cd52640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba740037a0 0x7fba74003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:09.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.912+0000 7fba8d553640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba74004ba0 0x7fba74004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:09.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.913+0000 7fba8dd54640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba74004160 0x7fba7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:09.916 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.913+0000 7fba8d553640 1 -- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba740037a0 msgr2=0x7fba74003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:09.916 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.913+0000 7fba8d553640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba740037a0 0x7fba74003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:09.916 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.913+0000 7fba8d553640 1 -- 172.21.15.7:0/2962287126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba74004160 msgr2=0x7fba7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:09.916 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.913+0000 7fba8d553640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba74004160 0x7fba7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:09.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.913+0000 7fba8d553640 1 -- 172.21.15.7:0/2962287126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba880ba400 con 0x7fba74004ba0 2024-09-15T23:36:09.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.914+0000 7fba8cd52640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba740037a0 0x7fba74003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:09.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.914+0000 7fba8dd54640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba74004160 0x7fba7400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:09.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.914+0000 7fba8dd54640 1 -- 172.21.15.7:0/2962287126 reap_dead start 2024-09-15T23:36:09.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.915+0000 7fba8d553640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba74004ba0 0x7fba74004fe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fba84009ac0 tx=0x7fba74005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b82cb62c1e27f6d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:09.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.915+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84036070 con 0x7fba74004ba0 2024-09-15T23:36:09.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.915+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba84031400 con 0x7fba74004ba0 2024-09-15T23:36:09.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.915+0000 7fba7ffff640 1 -- 172.21.15.7:0/2962287126 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84009690 con 0x7fba74004ba0 2024-09-15T23:36:09.918 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.916+0000 7fba8f7b7640 1 -- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba74004ba0 msgr2=0x7fba74004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:09.918 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.916+0000 7fba8f7b7640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba74004ba0 0x7fba74004fe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fba84009ac0 tx=0x7fba74005830 comp rx=0 tx=0).stop 2024-09-15T23:36:09.918 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.916+0000 7fba8f7b7640 1 -- 172.21.15.7:0/2962287126 shutdown_connections 2024-09-15T23:36:09.918 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.916+0000 7fba8f7b7640 1 --2- 172.21.15.7:0/2962287126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba74004ba0 0x7fba74004fe0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:09.918 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.916+0000 7fba8f7b7640 1 -- 172.21.15.7:0/2962287126 >> 172.21.15.7:0/2962287126 conn(0x7fba880d17f0 msgr2=0x7fba880d4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:09.918 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.917+0000 7fba8f7b7640 1 -- 172.21.15.7:0/2962287126 shutdown_connections 2024-09-15T23:36:09.918 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.917+0000 7fba8f7b7640 1 -- 172.21.15.7:0/2962287126 wait complete. 2024-09-15T23:36:09.920 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.919+0000 7fba8f7b7640 1 Processor -- start 2024-09-15T23:36:09.920 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.919+0000 7fba8f7b7640 1 -- start start 2024-09-15T23:36:09.922 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.921+0000 7fba8f7b7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba74004ba0 0x7fba881656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:09.922 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.921+0000 7fba8f7b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba881f9950 0x7fba88072590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:09.922 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.921+0000 7fba8f7b7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba88072ad0 0x7fba880c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:09.922 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.921+0000 7fba8f7b7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba88166620 con 0x7fba881f9950 2024-09-15T23:36:09.922 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.921+0000 7fba8f7b7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba880cf1d0 con 0x7fba88072ad0 2024-09-15T23:36:09.922 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.921+0000 7fba8f7b7640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fba881667a0 con 0x7fba74004ba0 2024-09-15T23:36:09.922 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.921+0000 7fba8d553640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba74004ba0 0x7fba881656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:09.922 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.921+0000 7fba8d553640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba74004ba0 0x7fba881656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:47812/0 (socket says 172.21.15.7:47812) 2024-09-15T23:36:09.923 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.922+0000 7fba8d553640 1 -- 172.21.15.7:0/398881850 learned_addr learned my addr 172.21.15.7:0/398881850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:09.923 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.922+0000 7fba8cd52640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba881f9950 0x7fba88072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.922+0000 7fba8dd54640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba88072ad0 0x7fba880c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.922+0000 7fba8dd54640 1 -- 172.21.15.7:0/398881850 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba74004ba0 msgr2=0x7fba881656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.922+0000 7fba8dd54640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba74004ba0 0x7fba881656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.922+0000 7fba8dd54640 1 -- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba881f9950 msgr2=0x7fba88072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.922+0000 7fba8dd54640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba881f9950 0x7fba88072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.922+0000 7fba8dd54640 1 -- 172.21.15.7:0/398881850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8806fd10 con 0x7fba88072ad0 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.923+0000 7fba8cd52640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba881f9950 0x7fba88072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.923+0000 7fba8dd54640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba88072ad0 0x7fba880c90b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba881678b0 tx=0x7fba800079d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.923+0000 7fba7dffb640 1 -- 172.21.15.7:0/398881850 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8000c180 con 0x7fba88072ad0 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.923+0000 7fba7dffb640 1 -- 172.21.15.7:0/398881850 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8000cb00 con 0x7fba88072ad0 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.923+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8806fff0 con 0x7fba88072ad0 2024-09-15T23:36:09.924 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.923+0000 7fba7dffb640 1 -- 172.21.15.7:0/398881850 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8000ccd0 con 0x7fba88072ad0 2024-09-15T23:36:09.925 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.923+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba880705b0 con 0x7fba88072ad0 2024-09-15T23:36:09.928 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.926+0000 7fba7dffb640 1 -- 172.21.15.7:0/398881850 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) ==== 90790+0+0 (secure 0 0 0) 0x7fba8001a020 con 0x7fba88072ad0 2024-09-15T23:36:09.928 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.926+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba880d4410 con 0x7fba88072ad0 2024-09-15T23:36:09.928 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.926+0000 7fba7dffb640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fba6c06c220 0x7fba6c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:09.929 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.927+0000 7fba8d553640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fba6c06c220 0x7fba6c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:09.929 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.928+0000 7fba8d553640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fba6c06c220 0x7fba6c06e6e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fba84009ac0 tx=0x7fba84008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:09.929 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.928+0000 7fba7dffb640 1 -- 172.21.15.7:0/398881850 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3726+0+0 (secure 0 0 0) 0x7fba800894f0 con 0x7fba88072ad0 2024-09-15T23:36:09.934 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:09.934+0000 7fba7dffb640 1 -- 172.21.15.7:0/398881850 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fba80059700 con 0x7fba88072ad0 2024-09-15T23:36:10.302 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.302+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba880b6430 con 0x7fba88072ad0 2024-09-15T23:36:10.303 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.303+0000 7fba7dffb640 1 -- 172.21.15.7:0/398881850 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10877 (secure 0 0 0) 0x7fba80059700 con 0x7fba88072ad0 2024-09-15T23:36:10.304 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-15T23:36:10.304 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":14,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:09.861159+0000","last_up_change":"2024-09-15T23:36:09.861159+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":7.690000057220459,"score_stable":7.690000057220459,"optimal_score":0.12999999523162842,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:10.310 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.310+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fba6c06c220 msgr2=0x7fba6c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:10.311 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.310+0000 7fba8f7b7640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fba6c06c220 0x7fba6c06e6e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fba84009ac0 tx=0x7fba84008030 comp rx=0 tx=0).stop 2024-09-15T23:36:10.311 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.310+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba88072ad0 msgr2=0x7fba880c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:10.311 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.310+0000 7fba8f7b7640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba88072ad0 0x7fba880c90b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba881678b0 tx=0x7fba800079d0 comp rx=0 tx=0).stop 2024-09-15T23:36:10.314 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.314+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 shutdown_connections 2024-09-15T23:36:10.314 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.314+0000 7fba8f7b7640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba88072ad0 0x7fba880c90b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:10.314 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.314+0000 7fba8f7b7640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba881f9950 0x7fba88072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:10.315 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.314+0000 7fba8f7b7640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fba6c06c220 0x7fba6c06e6e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:10.315 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.314+0000 7fba8f7b7640 1 --2- 172.21.15.7:0/398881850 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba74004ba0 0x7fba881656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:10.315 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.314+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 >> 172.21.15.7:0/398881850 conn(0x7fba880d17f0 msgr2=0x7fba880d3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:10.315 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.314+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 shutdown_connections 2024-09-15T23:36:10.315 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:10.314+0000 7fba8f7b7640 1 -- 172.21.15.7:0/398881850 wait complete. 2024-09-15T23:36:10.328 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-15T23:36:10.327+0000 7fa6e9c86640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T23:36:10.331 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-09-15T23:36:10.334 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-15T23:36:10.332+0000 7f11db34b640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T23:36:10.347 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-15T23:36:10.347+0000 7fa6e1aa6640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:36:10.350 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-15T23:36:10.350+0000 7f11d2957640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:36:10.384 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-15T23:36:10.383+0000 7f6d282c9640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T23:36:10.398 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-15T23:36:10.398+0000 7f6d200e9640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:36:10.414 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-15T23:36:10.413+0000 7f397863f640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T23:36:10.427 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-15T23:36:10.427+0000 7f397045f640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:36:17.138 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:17.243 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.242+0000 7f96cb4c9640 1 Processor -- start 2024-09-15T23:36:17.243 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.242+0000 7f96cb4c9640 1 -- start start 2024-09-15T23:36:17.244 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96cb4c9640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f96c415e7f0 0x7f96c40d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96cb4c9640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96c40d16e0 0x7f96c40b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96cb4c9640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c40d4430 con 0x7f96c40d36f0 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96cb4c9640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f96c40b7850 con 0x7f96c40d16e0 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96cb4c9640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f96c40b7b10 con 0x7f96c415e7f0 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96c9a66640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96c40d16e0 0x7f96c40b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96c8a64640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f96c40d36f0 0x7f96c40d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42230/0 (socket says 172.21.15.7:42230) 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96c9a66640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96c40d16e0 0x7f96c40b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49582/0 (socket says 172.21.15.7:49582) 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96c8a64640 1 -- 172.21.15.7:0/2379948600 learned_addr learned my addr 172.21.15.7:0/2379948600 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.243+0000 7f96c9265640 1 --2- 172.21.15.7:0/2379948600 >> v2:172.21.15.18:3300/0 conn(0x7f96c415e7f0 0x7f96c40d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.244+0000 7f96c9a66640 1 -- 172.21.15.7:0/2379948600 >> v2:172.21.15.18:3300/0 conn(0x7f96c415e7f0 msgr2=0x7f96c40d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.244+0000 7f96c9a66640 1 --2- 172.21.15.7:0/2379948600 >> v2:172.21.15.18:3300/0 conn(0x7f96c415e7f0 0x7f96c40d31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.244+0000 7f96c9a66640 1 -- 172.21.15.7:0/2379948600 >> v1:172.21.15.7:6789/0 conn(0x7f96c40d36f0 legacy=0x7f96c40d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.245 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.244+0000 7f96c9a66640 1 -- 172.21.15.7:0/2379948600 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c415ebd0 con 0x7f96c40d16e0 2024-09-15T23:36:17.246 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.244+0000 7f96bbfff640 1 -- 172.21.15.7:0/2379948600 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3801084922 0 0) 0x7f96c40d4430 con 0x7f96c40d36f0 2024-09-15T23:36:17.247 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.244+0000 7f96c9a66640 1 --2- 172.21.15.7:0/2379948600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c40d16e0 0x7f96c40b7fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f96c408d890 tx=0x7f96bc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601c0acba1a2ec27 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:17.247 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96bbfff640 1 -- 172.21.15.7:0/2379948600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc007740 con 0x7f96c40d16e0 2024-09-15T23:36:17.247 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96bbfff640 1 -- 172.21.15.7:0/2379948600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96bc0078e0 con 0x7f96c40d16e0 2024-09-15T23:36:17.247 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96bbfff640 1 -- 172.21.15.7:0/2379948600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc007cb0 con 0x7f96c40d16e0 2024-09-15T23:36:17.247 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96cb4c9640 1 -- 172.21.15.7:0/2379948600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c40d16e0 msgr2=0x7f96c40b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.247 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96cb4c9640 1 --2- 172.21.15.7:0/2379948600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c40d16e0 0x7f96c40b7fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f96c408d890 tx=0x7f96bc02a680 comp rx=0 tx=0).stop 2024-09-15T23:36:17.247 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96cb4c9640 1 -- 172.21.15.7:0/2379948600 shutdown_connections 2024-09-15T23:36:17.247 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96cb4c9640 1 --2- 172.21.15.7:0/2379948600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c40d16e0 0x7f96c40b7fa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.248 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96cb4c9640 1 --2- 172.21.15.7:0/2379948600 >> v2:172.21.15.18:3300/0 conn(0x7f96c415e7f0 0x7f96c40d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.248 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96cb4c9640 1 -- 172.21.15.7:0/2379948600 >> 172.21.15.7:0/2379948600 conn(0x7f96c406cfc0 msgr2=0x7f96c406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:17.248 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96cb4c9640 1 -- 172.21.15.7:0/2379948600 shutdown_connections 2024-09-15T23:36:17.248 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.245+0000 7f96cb4c9640 1 -- 172.21.15.7:0/2379948600 wait complete. 2024-09-15T23:36:17.248 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.246+0000 7f96cb4c9640 1 Processor -- start 2024-09-15T23:36:17.248 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96cb4c9640 1 -- start start 2024-09-15T23:36:17.250 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96cb4c9640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96c40d16e0 0x7f96c4155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96cb4c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96c40d36f0 0x7f96c4142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96cb4c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c415e7f0 0x7f96c40c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96cb4c9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40d4150 con 0x7f96c415e7f0 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96cb4c9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40d3e70 con 0x7f96c40d36f0 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96cb4c9640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f96c40cfc40 con 0x7f96c40d16e0 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96c9a66640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c415e7f0 0x7f96c40c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96c9a66640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c415e7f0 0x7f96c40c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49588/0 (socket says 172.21.15.7:49588) 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.247+0000 7f96c9a66640 1 -- 172.21.15.7:0/3052392462 learned_addr learned my addr 172.21.15.7:0/3052392462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96c8a64640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96c40d36f0 0x7f96c4142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96c9265640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96c40d16e0 0x7f96c4155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96c9a66640 1 -- 172.21.15.7:0/3052392462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96c40d16e0 msgr2=0x7f96c4155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.251 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96c9a66640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96c40d16e0 0x7f96c4155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96c9a66640 1 -- 172.21.15.7:0/3052392462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96c40d36f0 msgr2=0x7f96c4142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96c9a66640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96c40d36f0 0x7f96c4142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96c9a66640 1 -- 172.21.15.7:0/3052392462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c41f4160 con 0x7f96c415e7f0 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96c9265640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96c40d16e0 0x7f96c4155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96c9a66640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c415e7f0 0x7f96c40c9440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96bc02ac70 tx=0x7f96bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96b9ffb640 1 -- 172.21.15.7:0/3052392462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc040070 con 0x7f96c415e7f0 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96b9ffb640 1 -- 172.21.15.7:0/3052392462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96bc031bc0 con 0x7f96c415e7f0 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c41588d0 con 0x7f96c415e7f0 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96b9ffb640 1 -- 172.21.15.7:0/3052392462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc009450 con 0x7f96c415e7f0 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.248+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c40c9d10 con 0x7f96c415e7f0 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.249+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c40c9f10 con 0x7f96c415e7f0 2024-09-15T23:36:17.252 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.250+0000 7f96b9ffb640 1 -- 172.21.15.7:0/3052392462 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f96bc039040 con 0x7f96c415e7f0 2024-09-15T23:36:17.253 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.250+0000 7f96b9ffb640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f96a806c2e0 0x7f96a806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.253 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.251+0000 7f96b9ffb640 1 -- 172.21.15.7:0/3052392462 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f96bc0aca10 con 0x7f96c415e7f0 2024-09-15T23:36:17.253 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.251+0000 7f96c9265640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f96a806c2e0 0x7f96a806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.253 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.251+0000 7f96c9265640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f96a806c2e0 0x7f96a806e7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009af0 tx=0x7f96c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:17.256 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.255+0000 7f96b9ffb640 1 -- 172.21.15.7:0/3052392462 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f96bc07b3a0 con 0x7f96c415e7f0 2024-09-15T23:36:17.433 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.432+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96c40d31b0 con 0x7f96c415e7f0 2024-09-15T23:36:17.434 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.433+0000 7f96b9ffb640 1 -- 172.21.15.7:0/3052392462 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12801 (secure 0 0 0) 0x7f96bc07aca0 con 0x7f96c415e7f0 2024-09-15T23:36:17.434 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-15T23:36:17.434 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":17,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:12.932431+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:17.436 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f96a806c2e0 msgr2=0x7f96a806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.436 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f96a806c2e0 0x7f96a806e7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009af0 tx=0x7f96c0008040 comp rx=0 tx=0).stop 2024-09-15T23:36:17.436 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c415e7f0 msgr2=0x7f96c40c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.436 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c415e7f0 0x7f96c40c9440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96bc02ac70 tx=0x7f96bc004a40 comp rx=0 tx=0).stop 2024-09-15T23:36:17.436 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 shutdown_connections 2024-09-15T23:36:17.437 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c415e7f0 0x7f96c40c9440 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.437 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96c40d36f0 0x7f96c4142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.437 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f96a806c2e0 0x7f96a806e7a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.437 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 --2- 172.21.15.7:0/3052392462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f96c40d16e0 0x7f96c4155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.437 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 >> 172.21.15.7:0/3052392462 conn(0x7f96c406cfc0 msgr2=0x7f96c415d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:17.437 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 shutdown_connections 2024-09-15T23:36:17.437 INFO:teuthology.misc.health.smithi007.stderr:2024-09-15T23:36:17.435+0000 7f96cb4c9640 1 -- 172.21.15.7:0/3052392462 wait complete. 2024-09-15T23:36:17.445 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-15T23:36:17.445 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T23:36:17.446 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-15T23:36:17.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-15T23:36:17.448 INFO:tasks.ceph.mds.a:Started 2024-09-15T23:36:17.448 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-15T23:36:17.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-15T23:36:17.450 INFO:tasks.ceph.mds.c:Started 2024-09-15T23:36:17.450 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-15T23:36:17.451 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-15T23:36:17.453 INFO:tasks.ceph.mds.b:Started 2024-09-15T23:36:17.454 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-15T23:36:17.454 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-15T23:36:17.456 INFO:tasks.ceph.mds.d:Started 2024-09-15T23:36:17.456 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-15T23:36:17.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:36:17.525 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-15T23:36:17.526 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-15T23:36:17.543 INFO:tasks.ceph.mds.d.smithi018.stdout:starting mds.d at 2024-09-15T23:36:17.545 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2024-09-15T23:36:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.563+0000 7f5b3aca5640 1 Processor -- start 2024-09-15T23:36:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.563+0000 7f5b3aca5640 1 -- start start 2024-09-15T23:36:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.563+0000 7f5b3aca5640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5b3406c070 0x7f5b340aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.563+0000 7f5b3aca5640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3415ed90 con 0x7f5b34158bb0 2024-09-15T23:36:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b3aca5640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3415f070 con 0x7f5b3406bc10 2024-09-15T23:36:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b3aca5640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f5b340b2dd0 con 0x7f5b3406c070 2024-09-15T23:36:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b38a41640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5b34158bb0 0x7f5b340b26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42246/0 (socket says 172.21.15.7:42246) 2024-09-15T23:36:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b38a41640 1 -- 172.21.15.7:0/1188297087 learned_addr learned my addr 172.21.15.7:0/1188297087 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b39242640 1 --2- 172.21.15.7:0/1188297087 >> v2:172.21.15.18:3300/0 conn(0x7f5b3406c070 0x7f5b340aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b337fe640 1 -- 172.21.15.7:0/1188297087 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 528857499 0 0) 0x7f5b3415ed90 con 0x7f5b34158bb0 2024-09-15T23:36:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b337fe640 1 -- 172.21.15.7:0/1188297087 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b20003520 con 0x7f5b34158bb0 2024-09-15T23:36:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b39242640 1 -- 172.21.15.7:0/1188297087 >> v1:172.21.15.18:6789/0 conn(0x7f5b3406bc10 legacy=0x7f5b340ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b39242640 1 -- 172.21.15.7:0/1188297087 >> v1:172.21.15.7:6789/0 conn(0x7f5b34158bb0 legacy=0x7f5b340b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.564+0000 7f5b39242640 1 -- 172.21.15.7:0/1188297087 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3415f350 con 0x7f5b3406c070 2024-09-15T23:36:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.565+0000 7f5b39242640 1 --2- 172.21.15.7:0/1188297087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b3406c070 0x7f5b340aea40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0097a0 tx=0x7f5b2c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5846e8914d830fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.565+0000 7f5b337fe640 1 -- 172.21.15.7:0/1188297087 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c037070 con 0x7f5b3406c070 2024-09-15T23:36:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.565+0000 7f5b337fe640 1 -- 172.21.15.7:0/1188297087 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2c032440 con 0x7f5b3406c070 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.565+0000 7f5b337fe640 1 -- 172.21.15.7:0/1188297087 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c032810 con 0x7f5b3406c070 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.565+0000 7f5b3aca5640 1 -- 172.21.15.7:0/1188297087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b3406c070 msgr2=0x7f5b340aea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.565+0000 7f5b3aca5640 1 --2- 172.21.15.7:0/1188297087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b3406c070 0x7f5b340aea40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0097a0 tx=0x7f5b2c02a1a0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.565+0000 7f5b3aca5640 1 -- 172.21.15.7:0/1188297087 shutdown_connections 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.566+0000 7f5b3aca5640 1 --2- 172.21.15.7:0/1188297087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b3406c070 0x7f5b340aea40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.566+0000 7f5b3aca5640 1 -- 172.21.15.7:0/1188297087 >> 172.21.15.7:0/1188297087 conn(0x7f5b340b6990 msgr2=0x7f5b3409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.566+0000 7f5b3aca5640 1 -- 172.21.15.7:0/1188297087 shutdown_connections 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.566+0000 7f5b3aca5640 1 -- 172.21.15.7:0/1188297087 wait complete. 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.567+0000 7f5b3aca5640 1 Processor -- start 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.567+0000 7f5b3aca5640 1 -- start start 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b3aca5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3406bc10 0x7f5b340b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b3aca5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b3406c070 0x7f5b341535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b3aca5640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b34158bb0 0x7f5b3414a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b3aca5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3406c4c0 con 0x7f5b3406bc10 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b3aca5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3415eb50 con 0x7f5b3406c070 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b3aca5640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5b34158f90 con 0x7f5b34158bb0 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b38a41640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3406bc10 0x7f5b340b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b38a41640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3406bc10 0x7f5b340b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49634/0 (socket says 172.21.15.7:49634) 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b38a41640 1 -- 172.21.15.7:0/2140648587 learned_addr learned my addr 172.21.15.7:0/2140648587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b39242640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b34158bb0 0x7f5b3414a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b33fff640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b3406c070 0x7f5b341535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b38a41640 1 -- 172.21.15.7:0/2140648587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b34158bb0 msgr2=0x7f5b3414a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b38a41640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b34158bb0 0x7f5b3414a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b38a41640 1 -- 172.21.15.7:0/2140648587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b3406c070 msgr2=0x7f5b341535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b38a41640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b3406c070 0x7f5b341535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b38a41640 1 -- 172.21.15.7:0/2140648587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b34153ed0 con 0x7f5b3406bc10 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b38a41640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3406bc10 0x7f5b340b0e80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c410 tx=0x7f5b2400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b317fa640 1 -- 172.21.15.7:0/2140648587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24017070 con 0x7f5b3406bc10 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b317fa640 1 -- 172.21.15.7:0/2140648587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b24005ce0 con 0x7f5b3406bc10 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b317fa640 1 -- 172.21.15.7:0/2140648587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b240072f0 con 0x7f5b3406bc10 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b341541b0 con 0x7f5b3406bc10 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.568+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b34154a50 con 0x7f5b3406bc10 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.569+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b341545a0 con 0x7f5b3406bc10 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.570+0000 7f5b317fa640 1 -- 172.21.15.7:0/2140648587 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5b2401a070 con 0x7f5b3406bc10 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.570+0000 7f5b317fa640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5b1406c2b0 0x7f5b1406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.571+0000 7f5b317fa640 1 -- 172.21.15.7:0/2140648587 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5b2408a550 con 0x7f5b3406bc10 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.571+0000 7f5b33fff640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5b1406c2b0 0x7f5b1406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.571+0000 7f5b33fff640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5b1406c2b0 0x7f5b1406e770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c007fd0 tx=0x7f5b1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.575+0000 7f5b317fa640 1 -- 172.21.15.7:0/2140648587 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5b240590d0 con 0x7f5b3406bc10 2024-09-15T23:36:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.786+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b3406c8d0 con 0x7f5b3406bc10 2024-09-15T23:36:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.787+0000 7f5b317fa640 1 -- 172.21.15.7:0/2140648587 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1428 (secure 0 0 0) 0x7f5b240589d0 con 0x7f5b3406bc10 2024-09-15T23:36:17.788 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:17.788 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"btime":"2024-09-15T23:36:17:530160+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-15T23:36:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5b1406c2b0 msgr2=0x7f5b1406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5b1406c2b0 0x7f5b1406e770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c007fd0 tx=0x7f5b1c009040 comp rx=0 tx=0).stop 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3406bc10 msgr2=0x7f5b340b0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3406bc10 0x7f5b340b0e80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c410 tx=0x7f5b2400c8d0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 shutdown_connections 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5b1406c2b0 0x7f5b1406e770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b34158bb0 0x7f5b3414a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b3406c070 0x7f5b341535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 --2- 172.21.15.7:0/2140648587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3406bc10 0x7f5b340b0e80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 >> 172.21.15.7:0/2140648587 conn(0x7f5b340b6990 msgr2=0x7f5b34157170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 shutdown_connections 2024-09-15T23:36:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:17.789+0000 7f5b3aca5640 1 -- 172.21.15.7:0/2140648587 wait complete. 2024-09-15T23:36:19.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:36:19.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.915+0000 7f9680123640 1 Processor -- start 2024-09-15T23:36:19.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.916+0000 7f9680123640 1 -- start start 2024-09-15T23:36:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.916+0000 7f9680123640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f967809acb0 0x7f96780b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.916+0000 7f9680123640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96780b3990 con 0x7f967815eda0 2024-09-15T23:36:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.916+0000 7f9680123640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96780b3c30 con 0x7f967809b090 2024-09-15T23:36:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.916+0000 7f9680123640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f967809b800 con 0x7f967809acb0 2024-09-15T23:36:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.917+0000 7f967debf640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f967809acb0 0x7f96780b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.917+0000 7f967debf640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f967809acb0 0x7f96780b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49638/0 (socket says 172.21.15.7:49638) 2024-09-15T23:36:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.917+0000 7f967debf640 1 -- 172.21.15.7:0/748915216 learned_addr learned my addr 172.21.15.7:0/748915216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.917+0000 7f967e6c0640 1 --1- 172.21.15.7:0/748915216 >> v1:172.21.15.7:6789/0 conn(0x7f967815eda0 0x7f96780b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42254/0 (socket says 172.21.15.7:42254) 2024-09-15T23:36:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.917+0000 7f967debf640 1 -- 172.21.15.7:0/748915216 >> v1:172.21.15.18:6789/0 conn(0x7f967809b090 legacy=0x7f967815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.917+0000 7f967debf640 1 -- 172.21.15.7:0/748915216 >> v1:172.21.15.7:6789/0 conn(0x7f967815eda0 legacy=0x7f96780b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.917+0000 7f967cebd640 1 -- 172.21.15.7:0/748915216 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519552394 0 0) 0x7f96780b3990 con 0x7f967815eda0 2024-09-15T23:36:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.917+0000 7f967debf640 1 -- 172.21.15.7:0/748915216 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96780d1000 con 0x7f967809acb0 2024-09-15T23:36:19.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.918+0000 7f967debf640 1 --2- 172.21.15.7:0/748915216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967809acb0 0x7f96780b32b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9668009870 tx=0x7f966802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6166faf1da2fca76 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:19.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.918+0000 7f967cebd640 1 -- 172.21.15.7:0/748915216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668037070 con 0x7f967809acb0 2024-09-15T23:36:19.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.918+0000 7f967cebd640 1 -- 172.21.15.7:0/748915216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9668032440 con 0x7f967809acb0 2024-09-15T23:36:19.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.918+0000 7f967cebd640 1 -- 172.21.15.7:0/748915216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668032810 con 0x7f967809acb0 2024-09-15T23:36:19.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.918+0000 7f9680123640 1 -- 172.21.15.7:0/748915216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967809acb0 msgr2=0x7f96780b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:19.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.918+0000 7f9680123640 1 --2- 172.21.15.7:0/748915216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967809acb0 0x7f96780b32b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9668009870 tx=0x7f966802a270 comp rx=0 tx=0).stop 2024-09-15T23:36:19.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.918+0000 7f9680123640 1 -- 172.21.15.7:0/748915216 shutdown_connections 2024-09-15T23:36:19.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.918+0000 7f9680123640 1 --2- 172.21.15.7:0/748915216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967809acb0 0x7f96780b32b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:19.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.918+0000 7f9680123640 1 -- 172.21.15.7:0/748915216 >> 172.21.15.7:0/748915216 conn(0x7f96780d2410 msgr2=0x7f96780d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:19.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.919+0000 7f9680123640 1 -- 172.21.15.7:0/748915216 shutdown_connections 2024-09-15T23:36:19.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.919+0000 7f9680123640 1 -- 172.21.15.7:0/748915216 wait complete. 2024-09-15T23:36:19.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.920+0000 7f9680123640 1 Processor -- start 2024-09-15T23:36:19.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.920+0000 7f9680123640 1 -- start start 2024-09-15T23:36:19.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f9680123640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f967809acb0 0x7f9678159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:19.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f9680123640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f967809b090 0x7f967806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:19.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f9680123640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967815eda0 0x7f96780c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:19.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f9680123640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96780d2f30 con 0x7f967815eda0 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f9680123640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f96780d1370 con 0x7f967809b090 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f9680123640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f96780d30b0 con 0x7f967809acb0 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967e6c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967815eda0 0x7f96780c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967e6c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967815eda0 0x7f96780c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49644/0 (socket says 172.21.15.7:49644) 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967e6c0640 1 -- 172.21.15.7:0/3754581118 learned_addr learned my addr 172.21.15.7:0/3754581118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967debf640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f967809acb0 0x7f9678159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967d6be640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f967809b090 0x7f967806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967e6c0640 1 -- 172.21.15.7:0/3754581118 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f967809acb0 msgr2=0x7f9678159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967e6c0640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f967809acb0 0x7f9678159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967e6c0640 1 -- 172.21.15.7:0/3754581118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f967809b090 msgr2=0x7f967806dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967e6c0640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f967809b090 0x7f967806dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:19.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967e6c0640 1 -- 172.21.15.7:0/3754581118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96780c7ad0 con 0x7f967815eda0 2024-09-15T23:36:19.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f967e6c0640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967815eda0 0x7f96780c8890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f967000c730 tx=0x7f967000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:19.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f96767fc640 1 -- 172.21.15.7:0/3754581118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670018070 con 0x7f967815eda0 2024-09-15T23:36:19.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f96767fc640 1 -- 172.21.15.7:0/3754581118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967000ce40 con 0x7f967815eda0 2024-09-15T23:36:19.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96780c7db0 con 0x7f967815eda0 2024-09-15T23:36:19.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f96767fc640 1 -- 172.21.15.7:0/3754581118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967000a860 con 0x7f967815eda0 2024-09-15T23:36:19.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.921+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96780cc700 con 0x7f967815eda0 2024-09-15T23:36:19.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.922+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96780b32b0 con 0x7f967815eda0 2024-09-15T23:36:19.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.923+0000 7f96767fc640 1 -- 172.21.15.7:0/3754581118 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f967001b070 con 0x7f967815eda0 2024-09-15T23:36:19.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.923+0000 7f96767fc640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f966406c2b0 0x7f966406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:19.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.924+0000 7f96767fc640 1 -- 172.21.15.7:0/3754581118 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f967008a500 con 0x7f967815eda0 2024-09-15T23:36:19.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.924+0000 7f967debf640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f966406c2b0 0x7f966406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:19.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.924+0000 7f967debf640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f966406c2b0 0x7f966406e770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9668034000 tx=0x7f966802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:19.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:19.928+0000 7f96767fc640 1 -- 172.21.15.7:0/3754581118 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9670058a00 con 0x7f967815eda0 2024-09-15T23:36:20.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.139+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96780b8370 con 0x7f967815eda0 2024-09-15T23:36:20.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.139+0000 7f96767fc640 1 -- 172.21.15.7:0/3754581118 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3897 (secure 0 0 0) 0x7f967005cd00 con 0x7f967815eda0 2024-09-15T23:36:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:20.141 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"btime":"2024-09-15T23:36:18:531013+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T23:36:20.141 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-09-15T23:36:20.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.141+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f966406c2b0 msgr2=0x7f966406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.141+0000 7f9680123640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f966406c2b0 0x7f966406e770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9668034000 tx=0x7f966802e000 comp rx=0 tx=0).stop 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.141+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967815eda0 msgr2=0x7f96780c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.141+0000 7f9680123640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967815eda0 0x7f96780c8890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f967000c730 tx=0x7f967000cbf0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.141+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 shutdown_connections 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.141+0000 7f9680123640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967815eda0 0x7f96780c8890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.141+0000 7f9680123640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f967809b090 0x7f967806dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.142+0000 7f9680123640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f966406c2b0 0x7f966406e770 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.142+0000 7f9680123640 1 --2- 172.21.15.7:0/3754581118 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f967809acb0 0x7f9678159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.142+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 >> 172.21.15.7:0/3754581118 conn(0x7f96780d2410 msgr2=0x7f967815c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.142+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 shutdown_connections 2024-09-15T23:36:20.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.142+0000 7f9680123640 1 -- 172.21.15.7:0/3754581118 wait complete. 2024-09-15T23:36:20.153 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-15T23:36:20.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.255+0000 7ff6cccdd640 1 Processor -- start 2024-09-15T23:36:20.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.255+0000 7ff6cccdd640 1 -- start start 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.255+0000 7ff6cccdd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6c815e840 0x7ff6c80d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.255+0000 7ff6cccdd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c806dd80 con 0x7ff6c80d0f90 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.255+0000 7ff6cccdd640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c80cf560 con 0x7ff6c80d52e0 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.255+0000 7ff6cccdd640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c80cfe90 con 0x7ff6c815e840 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.256+0000 7ff6c659c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6c815e840 0x7ff6c80d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.256+0000 7ff6c6d9d640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff6c80d0f90 0x7ff6c80d24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42266/0 (socket says 172.21.15.7:42266) 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.256+0000 7ff6c6d9d640 1 -- 172.21.15.7:0/4001116396 learned_addr learned my addr 172.21.15.7:0/4001116396 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.256+0000 7ff6c659c640 1 -- 172.21.15.7:0/4001116396 >> v1:172.21.15.18:6789/0 conn(0x7ff6c80d52e0 legacy=0x7ff6c806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.256+0000 7ff6c659c640 1 -- 172.21.15.7:0/4001116396 >> v1:172.21.15.7:6789/0 conn(0x7ff6c80d0f90 legacy=0x7ff6c80d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.256+0000 7ff6c659c640 1 -- 172.21.15.7:0/4001116396 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c80cf840 con 0x7ff6c815e840 2024-09-15T23:36:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.256+0000 7ff6c5d9b640 1 -- 172.21.15.7:0/4001116396 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402612956 0 0) 0x7ff6c806dd80 con 0x7ff6c80d0f90 2024-09-15T23:36:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6c659c640 1 --2- 172.21.15.7:0/4001116396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c815e840 0x7ff6c80d0320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80097a0 tx=0x7ff6b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf1898e37367081 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6c5d9b640 1 -- 172.21.15.7:0/4001116396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8037070 con 0x7ff6c815e840 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6c5d9b640 1 -- 172.21.15.7:0/4001116396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b8032440 con 0x7ff6c815e840 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6c5d9b640 1 -- 172.21.15.7:0/4001116396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8032810 con 0x7ff6c815e840 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6cccdd640 1 -- 172.21.15.7:0/4001116396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c815e840 msgr2=0x7ff6c80d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6cccdd640 1 --2- 172.21.15.7:0/4001116396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c815e840 0x7ff6c80d0320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80097a0 tx=0x7ff6b802a230 comp rx=0 tx=0).stop 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6cccdd640 1 -- 172.21.15.7:0/4001116396 shutdown_connections 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6cccdd640 1 --2- 172.21.15.7:0/4001116396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c815e840 0x7ff6c80d0320 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6cccdd640 1 -- 172.21.15.7:0/4001116396 >> 172.21.15.7:0/4001116396 conn(0x7ff6c808bfd0 msgr2=0x7ff6c80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6cccdd640 1 -- 172.21.15.7:0/4001116396 shutdown_connections 2024-09-15T23:36:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.257+0000 7ff6cccdd640 1 -- 172.21.15.7:0/4001116396 wait complete. 2024-09-15T23:36:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.259+0000 7ff6cccdd640 1 Processor -- start 2024-09-15T23:36:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.259+0000 7ff6cccdd640 1 -- start start 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6cccdd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6c80d0f90 0x7ff6c815a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6cccdd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c80d52e0 0x7ff6c80ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6cccdd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6c815e840 0x7ff6c80ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6cccdd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c815ec20 con 0x7ff6c80d52e0 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6cccdd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c806e030 con 0x7ff6c80d0f90 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6cccdd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c80c8d00 con 0x7ff6c815e840 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c80d52e0 0x7ff6c80ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c80d52e0 0x7ff6c80ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49670/0 (socket says 172.21.15.7:49670) 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bdd9b640 1 -- 172.21.15.7:0/1287023419 learned_addr learned my addr 172.21.15.7:0/1287023419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6c659c640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6c80d0f90 0x7ff6c815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6c6d9d640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6c815e840 0x7ff6c80ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bdd9b640 1 -- 172.21.15.7:0/1287023419 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6c815e840 msgr2=0x7ff6c80ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bdd9b640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6c815e840 0x7ff6c80ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bdd9b640 1 -- 172.21.15.7:0/1287023419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6c80d0f90 msgr2=0x7ff6c815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bdd9b640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6c80d0f90 0x7ff6c815a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bdd9b640 1 -- 172.21.15.7:0/1287023419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c80c80f0 con 0x7ff6c80d52e0 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bdd9b640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c80d52e0 0x7ff6c80ca310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c450 tx=0x7ff6b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bf7fe640 1 -- 172.21.15.7:0/1287023419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00052d0 con 0x7ff6c80d52e0 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bf7fe640 1 -- 172.21.15.7:0/1287023419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b0005490 con 0x7ff6c80d52e0 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c80c83d0 con 0x7ff6c80d52e0 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6bf7fe640 1 -- 172.21.15.7:0/1287023419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0003880 con 0x7ff6c80d52e0 2024-09-15T23:36:20.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.260+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c80c8990 con 0x7ff6c80d52e0 2024-09-15T23:36:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.262+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c80d08a0 con 0x7ff6c80d52e0 2024-09-15T23:36:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.262+0000 7ff6bf7fe640 1 -- 172.21.15.7:0/1287023419 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff6b0003a20 con 0x7ff6c80d52e0 2024-09-15T23:36:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.262+0000 7ff6bf7fe640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff6a806c2e0 0x7ff6a806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.262+0000 7ff6bf7fe640 1 -- 172.21.15.7:0/1287023419 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff6b008ac10 con 0x7ff6c80d52e0 2024-09-15T23:36:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.262+0000 7ff6c659c640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff6a806c2e0 0x7ff6a806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.263+0000 7ff6c659c640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff6a806c2e0 0x7ff6a806e7a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8034000 tx=0x7ff6b802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:20.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.267+0000 7ff6bf7fe640 1 -- 172.21.15.7:0/1287023419 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff6b0058ff0 con 0x7ff6c80d52e0 2024-09-15T23:36:20.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.445+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff6c80d1cb0 con 0x7ff6c80d52e0 2024-09-15T23:36:20.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.445+0000 7ff6bf7fe640 1 -- 172.21.15.7:0/1287023419 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7ff6b005d2f0 con 0x7ff6c80d52e0 2024-09-15T23:36:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:20.446 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-15T23:36:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:36:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-15T23:36:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-15T23:36:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:36:20.447 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-15T23:36:20.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff6a806c2e0 msgr2=0x7ff6a806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff6a806c2e0 0x7ff6a806e7a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8034000 tx=0x7ff6b802e000 comp rx=0 tx=0).stop 2024-09-15T23:36:20.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c80d52e0 msgr2=0x7ff6c80ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c80d52e0 0x7ff6c80ca310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c450 tx=0x7ff6b000c910 comp rx=0 tx=0).stop 2024-09-15T23:36:20.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 shutdown_connections 2024-09-15T23:36:20.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff6a806c2e0 0x7ff6a806e7a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6c815e840 0x7ff6c80ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c80d52e0 0x7ff6c80ca310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 --2- 172.21.15.7:0/1287023419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6c80d0f90 0x7ff6c815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 >> 172.21.15.7:0/1287023419 conn(0x7ff6c808bfd0 msgr2=0x7ff6c80b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:20.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 shutdown_connections 2024-09-15T23:36:20.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.447+0000 7ff6cccdd640 1 -- 172.21.15.7:0/1287023419 wait complete. 2024-09-15T23:36:20.459 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-15T23:36:20.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.564+0000 7f6b0fa77640 1 Processor -- start 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.564+0000 7f6b0fa77640 1 -- start start 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.565+0000 7f6b0fa77640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d0530 0x7f6b080d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.565+0000 7f6b0fa77640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b080d2470 con 0x7f6b0815e6b0 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.565+0000 7f6b0fa77640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b080d2750 con 0x7f6b080d1390 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.565+0000 7f6b0fa77640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6b080d55c0 con 0x7f6b080d0530 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.565+0000 7f6b0d813640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6b0815e6b0 0x7f6b080d0c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42270/0 (socket says 172.21.15.7:42270) 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.565+0000 7f6b0d813640 1 -- 172.21.15.7:0/3357232428 learned_addr learned my addr 172.21.15.7:0/3357232428 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.565+0000 7f6b0e014640 1 --2- 172.21.15.7:0/3357232428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d0530 0x7f6b080d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.566+0000 7f6b0c811640 1 -- 172.21.15.7:0/3357232428 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898461105 0 0) 0x7f6b080d2470 con 0x7f6b0815e6b0 2024-09-15T23:36:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.566+0000 7f6b0c811640 1 -- 172.21.15.7:0/3357232428 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6af0003520 con 0x7f6b0815e6b0 2024-09-15T23:36:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.566+0000 7f6b0c811640 1 -- 172.21.15.7:0/3357232428 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3893620263 0 0) 0x7f6b080d2750 con 0x7f6b080d1390 2024-09-15T23:36:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.566+0000 7f6b0c811640 1 -- 172.21.15.7:0/3357232428 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b080d2470 con 0x7f6b080d1390 2024-09-15T23:36:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.566+0000 7f6b0e014640 1 -- 172.21.15.7:0/3357232428 >> v1:172.21.15.18:6789/0 conn(0x7f6b080d1390 legacy=0x7f6b080cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.566+0000 7f6b0e014640 1 -- 172.21.15.7:0/3357232428 >> v1:172.21.15.7:6789/0 conn(0x7f6b0815e6b0 legacy=0x7f6b080d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.566+0000 7f6b0e014640 1 -- 172.21.15.7:0/3357232428 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b080d2a30 con 0x7f6b080d0530 2024-09-15T23:36:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0e014640 1 --2- 172.21.15.7:0/3357232428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d0530 0x7f6b080d3230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6b000097d0 tx=0x7f6b0002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e61c450cfe2e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0c811640 1 -- 172.21.15.7:0/3357232428 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00037070 con 0x7f6b080d0530 2024-09-15T23:36:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0c811640 1 -- 172.21.15.7:0/3357232428 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b00032440 con 0x7f6b080d0530 2024-09-15T23:36:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0c811640 1 -- 172.21.15.7:0/3357232428 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00032810 con 0x7f6b080d0530 2024-09-15T23:36:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0fa77640 1 -- 172.21.15.7:0/3357232428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d0530 msgr2=0x7f6b080d3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0fa77640 1 --2- 172.21.15.7:0/3357232428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d0530 0x7f6b080d3230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6b000097d0 tx=0x7f6b0002a210 comp rx=0 tx=0).stop 2024-09-15T23:36:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0fa77640 1 -- 172.21.15.7:0/3357232428 shutdown_connections 2024-09-15T23:36:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0fa77640 1 --2- 172.21.15.7:0/3357232428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d0530 0x7f6b080d3230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0fa77640 1 -- 172.21.15.7:0/3357232428 >> 172.21.15.7:0/3357232428 conn(0x7f6b0808bfd0 msgr2=0x7f6b080d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.567+0000 7f6b0fa77640 1 -- 172.21.15.7:0/3357232428 shutdown_connections 2024-09-15T23:36:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.568+0000 7f6b0fa77640 1 -- 172.21.15.7:0/3357232428 wait complete. 2024-09-15T23:36:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.569+0000 7f6b0fa77640 1 Processor -- start 2024-09-15T23:36:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.569+0000 7f6b0fa77640 1 -- start start 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.569+0000 7f6b0fa77640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b080d0530 0x7f6b081f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.569+0000 7f6b0fa77640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d1390 0x7f6b080cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.569+0000 7f6b0fa77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b0815e6b0 0x7f6b080cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.569+0000 7f6b0fa77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0806cd40 con 0x7f6b0815e6b0 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.569+0000 7f6b0fa77640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6b080d2220 con 0x7f6b080d0530 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.569+0000 7f6b0fa77640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6b0806cec0 con 0x7f6b080d1390 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0e014640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b0815e6b0 0x7f6b080cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0e014640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b0815e6b0 0x7f6b080cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49682/0 (socket says 172.21.15.7:49682) 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0e014640 1 -- 172.21.15.7:0/1903745275 learned_addr learned my addr 172.21.15.7:0/1903745275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0d813640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b080d0530 0x7f6b081f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0d012640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d1390 0x7f6b080cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0e014640 1 -- 172.21.15.7:0/1903745275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d1390 msgr2=0x7f6b080cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0e014640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d1390 0x7f6b080cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0e014640 1 -- 172.21.15.7:0/1903745275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b080d0530 msgr2=0x7f6b081f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0e014640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b080d0530 0x7f6b081f7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0e014640 1 -- 172.21.15.7:0/1903745275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b080ccd10 con 0x7f6b0815e6b0 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0d012640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d1390 0x7f6b080cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0e014640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b0815e6b0 0x7f6b080cc630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b00035040 tx=0x7f6b000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b067fc640 1 -- 172.21.15.7:0/1903745275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00041020 con 0x7f6b0815e6b0 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b067fc640 1 -- 172.21.15.7:0/1903745275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0003d640 con 0x7f6b0815e6b0 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b067fc640 1 -- 172.21.15.7:0/1903745275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00037040 con 0x7f6b0815e6b0 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b081e3ae0 con 0x7f6b0815e6b0 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.570+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b081e40a0 con 0x7f6b0815e6b0 2024-09-15T23:36:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.571+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b080d3230 con 0x7f6b0815e6b0 2024-09-15T23:36:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.572+0000 7f6b067fc640 1 -- 172.21.15.7:0/1903745275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6b00044020 con 0x7f6b0815e6b0 2024-09-15T23:36:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.572+0000 7f6b067fc640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6af406c2b0 0x7f6af406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.572+0000 7f6b067fc640 1 -- 172.21.15.7:0/1903745275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f6b000ab710 con 0x7f6b0815e6b0 2024-09-15T23:36:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.572+0000 7f6b0d813640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6af406c2b0 0x7f6af406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.573+0000 7f6b0d813640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6af406c2b0 0x7f6af406e770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6af80012e0 tx=0x7f6af8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:20.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.577+0000 7f6b067fc640 1 -- 172.21.15.7:0/1903745275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b0004b030 con 0x7f6b0815e6b0 2024-09-15T23:36:20.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:20.750+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6b081e4270 con 0x7f6b0815e6b0 2024-09-15T23:36:21.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.539+0000 7f6b067fc640 1 -- 172.21.15.7:0/1903745275 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f6b0007a1e0 con 0x7f6b0815e6b0 2024-09-15T23:36:21.540 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6af406c2b0 msgr2=0x7f6af406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6af406c2b0 0x7f6af406e770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6af80012e0 tx=0x7f6af8009040 comp rx=0 tx=0).stop 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b0815e6b0 msgr2=0x7f6b080cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b0815e6b0 0x7f6b080cc630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b00035040 tx=0x7f6b000046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 shutdown_connections 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b0815e6b0 0x7f6b080cc630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6af406c2b0 0x7f6af406e770 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6b080d1390 0x7f6b080cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 --2- 172.21.15.7:0/1903745275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b080d0530 0x7f6b081f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 >> 172.21.15.7:0/1903745275 conn(0x7f6b0808bfd0 msgr2=0x7f6b0815d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 shutdown_connections 2024-09-15T23:36:21.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.542+0000 7f6b0fa77640 1 -- 172.21.15.7:0/1903745275 wait complete. 2024-09-15T23:36:21.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-15T23:36:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.676+0000 7fac6b642640 1 Processor -- start 2024-09-15T23:36:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.676+0000 7fac6b642640 1 -- start start 2024-09-15T23:36:21.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.677+0000 7fac6b642640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fac6415e5b0 0x7fac64162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.677+0000 7fac6b642640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fac640d24e0 0x7fac64162890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.677+0000 7fac6b642640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac640d2980 con 0x7fac6415e990 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.677+0000 7fac6b642640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fac640d2c40 con 0x7fac640d24e0 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.677+0000 7fac6b642640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fac640d2f20 con 0x7fac6415e5b0 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.677+0000 7fac68bdd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fac640d24e0 0x7fac64162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.677+0000 7fac69bdf640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fac6415e990 0x7fac640d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42280/0 (socket says 172.21.15.7:42280) 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.677+0000 7fac69bdf640 1 -- 172.21.15.7:0/746690912 learned_addr learned my addr 172.21.15.7:0/746690912 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.677+0000 7fac693de640 1 --2- 172.21.15.7:0/746690912 >> v2:172.21.15.18:3300/0 conn(0x7fac6415e5b0 0x7fac64162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.678+0000 7fac68bdd640 1 -- 172.21.15.7:0/746690912 >> v2:172.21.15.18:3300/0 conn(0x7fac6415e5b0 msgr2=0x7fac64162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.678+0000 7fac68bdd640 1 --2- 172.21.15.7:0/746690912 >> v2:172.21.15.18:3300/0 conn(0x7fac6415e5b0 0x7fac64162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.678+0000 7fac68bdd640 1 -- 172.21.15.7:0/746690912 >> v1:172.21.15.7:6789/0 conn(0x7fac6415e990 legacy=0x7fac640d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.678+0000 7fac68bdd640 1 -- 172.21.15.7:0/746690912 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac640d3200 con 0x7fac640d24e0 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.678+0000 7fac5bfff640 1 -- 172.21.15.7:0/746690912 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1923297264 0 0) 0x7fac640d2980 con 0x7fac6415e990 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.678+0000 7fac68bdd640 1 --2- 172.21.15.7:0/746690912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac640d24e0 0x7fac64162890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac54009790 tx=0x7fac5402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52bb9735e8ccefa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.678+0000 7fac5bfff640 1 -- 172.21.15.7:0/746690912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54037070 con 0x7fac640d24e0 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.678+0000 7fac5bfff640 1 -- 172.21.15.7:0/746690912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac54032440 con 0x7fac640d24e0 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.679+0000 7fac5bfff640 1 -- 172.21.15.7:0/746690912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54032810 con 0x7fac640d24e0 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.679+0000 7fac6b642640 1 -- 172.21.15.7:0/746690912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac640d24e0 msgr2=0x7fac64162890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.679+0000 7fac6b642640 1 --2- 172.21.15.7:0/746690912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac640d24e0 0x7fac64162890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac54009790 tx=0x7fac5402a1c0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.679+0000 7fac6b642640 1 -- 172.21.15.7:0/746690912 shutdown_connections 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.679+0000 7fac6b642640 1 --2- 172.21.15.7:0/746690912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac640d24e0 0x7fac64162890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.679+0000 7fac6b642640 1 --2- 172.21.15.7:0/746690912 >> v2:172.21.15.18:3300/0 conn(0x7fac6415e5b0 0x7fac64162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.679+0000 7fac6b642640 1 -- 172.21.15.7:0/746690912 >> 172.21.15.7:0/746690912 conn(0x7fac6408bfd0 msgr2=0x7fac640d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.679+0000 7fac6b642640 1 -- 172.21.15.7:0/746690912 shutdown_connections 2024-09-15T23:36:21.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.679+0000 7fac6b642640 1 -- 172.21.15.7:0/746690912 wait complete. 2024-09-15T23:36:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.680+0000 7fac6b642640 1 Processor -- start 2024-09-15T23:36:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.680+0000 7fac6b642640 1 -- start start 2024-09-15T23:36:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac6b642640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac640d24e0 0x7fac640b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac6b642640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6415e5b0 0x7fac641f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac6b642640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac6415e990 0x7fac641f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac6b642640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac640b3b40 con 0x7fac6415e5b0 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac6b642640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fac640b39c0 con 0x7fac6415e990 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac6b642640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fac640d0430 con 0x7fac640d24e0 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac68bdd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6415e5b0 0x7fac641f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac68bdd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6415e5b0 0x7fac641f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49696/0 (socket says 172.21.15.7:49696) 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac68bdd640 1 -- 172.21.15.7:0/1553010720 learned_addr learned my addr 172.21.15.7:0/1553010720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac693de640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac640d24e0 0x7fac640b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac69bdf640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac6415e990 0x7fac641f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.681+0000 7fac68bdd640 1 -- 172.21.15.7:0/1553010720 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac640d24e0 msgr2=0x7fac640b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac68bdd640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac640d24e0 0x7fac640b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac68bdd640 1 -- 172.21.15.7:0/1553010720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac6415e990 msgr2=0x7fac641f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac68bdd640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac6415e990 0x7fac641f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac68bdd640 1 -- 172.21.15.7:0/1553010720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac641f78d0 con 0x7fac6415e5b0 2024-09-15T23:36:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac693de640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac640d24e0 0x7fac640b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac68bdd640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6415e5b0 0x7fac641f5090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac54035040 tx=0x7fac540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac59ffb640 1 -- 172.21.15.7:0/1553010720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54037040 con 0x7fac6415e5b0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac59ffb640 1 -- 172.21.15.7:0/1553010720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac54005bd0 con 0x7fac6415e5b0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac640b7b10 con 0x7fac6415e5b0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac59ffb640 1 -- 172.21.15.7:0/1553010720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54006030 con 0x7fac6415e5b0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.682+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac640b80d0 con 0x7fac6415e5b0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.683+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac64162350 con 0x7fac6415e5b0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.684+0000 7fac59ffb640 1 -- 172.21.15.7:0/1553010720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fac540061d0 con 0x7fac6415e5b0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.684+0000 7fac59ffb640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fac4806c2b0 0x7fac4806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.684+0000 7fac59ffb640 1 -- 172.21.15.7:0/1553010720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fac540aba20 con 0x7fac6415e5b0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.684+0000 7fac693de640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fac4806c2b0 0x7fac4806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.685+0000 7fac693de640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fac4806c2b0 0x7fac4806e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac60009b90 tx=0x7fac60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:21.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.689+0000 7fac59ffb640 1 -- 172.21.15.7:0/1553010720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fac5407a370 con 0x7fac6415e5b0 2024-09-15T23:36:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:21.867+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fac640ba560 con 0x7fac6415e5b0 2024-09-15T23:36:22.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.541+0000 7fac59ffb640 1 -- 172.21.15.7:0/1553010720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7fac54080030 con 0x7fac6415e5b0 2024-09-15T23:36:22.542 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.546+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fac4806c2b0 msgr2=0x7fac4806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.546+0000 7fac6b642640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fac4806c2b0 0x7fac4806e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac60009b90 tx=0x7fac60008040 comp rx=0 tx=0).stop 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.546+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6415e5b0 msgr2=0x7fac641f5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.546+0000 7fac6b642640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6415e5b0 0x7fac641f5090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac54035040 tx=0x7fac540046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.547+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 shutdown_connections 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.547+0000 7fac6b642640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac6415e990 0x7fac641f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.547+0000 7fac6b642640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6415e5b0 0x7fac641f5090 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.547+0000 7fac6b642640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fac4806c2b0 0x7fac4806e770 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac60009b90 tx=0x7fac60008040 comp rx=0 tx=0).stop 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.547+0000 7fac6b642640 1 --2- 172.21.15.7:0/1553010720 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac640d24e0 0x7fac640b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.547+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 >> 172.21.15.7:0/1553010720 conn(0x7fac6408bfd0 msgr2=0x7fac640d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.552+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 shutdown_connections 2024-09-15T23:36:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.553+0000 7fac6b642640 1 -- 172.21.15.7:0/1553010720 wait complete. 2024-09-15T23:36:22.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-15T23:36:22.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.689+0000 7f99fdf6d640 1 Processor -- start 2024-09-15T23:36:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.690+0000 7f99fdf6d640 1 -- start start 2024-09-15T23:36:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.690+0000 7f99fdf6d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f815fea0 0x7f99f80b3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.690+0000 7f99fdf6d640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f80cde50 con 0x7f99f80d0f90 2024-09-15T23:36:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.690+0000 7f99fdf6d640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f80ce130 con 0x7f99f809bae0 2024-09-15T23:36:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.690+0000 7f99fdf6d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f99f80b4ed0 con 0x7f99f815fea0 2024-09-15T23:36:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f7fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f99f80d0f90 0x7f99f80c6210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42282/0 (socket says 172.21.15.7:42282) 2024-09-15T23:36:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f7fff640 1 -- 172.21.15.7:0/357849628 learned_addr learned my addr 172.21.15.7:0/357849628 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f77fe640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f815fea0 0x7f99f80b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1966249321 0 0) 0x7f99f80cde50 con 0x7f99f80d0f90 2024-09-15T23:36:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99e0003520 con 0x7f99f80d0f90 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1276909780 0 0) 0x7f99e0003520 con 0x7f99f80d0f90 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f815fea0 msgr2=0x7f99f80b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f815fea0 0x7f99f80b3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 >> v1:172.21.15.18:6789/0 conn(0x7f99f809bae0 legacy=0x7f99f80d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f80ce410 con 0x7f99f80d0f90 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f99f00019f0 con 0x7f99f80d0f90 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 >> v1:172.21.15.7:6789/0 conn(0x7f99f80d0f90 legacy=0x7f99f80c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99e00039e0 0x7f99e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f815fea0 0x7f99e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99f80b4ed0 con 0x7f99e0004d40 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99e000ba70 con 0x7f99e00039e0 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f99e000bc70 con 0x7f99f815fea0 2024-09-15T23:36:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f7fff640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f77fe640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99e00039e0 0x7f99e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.691+0000 7f99f6ffd640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f815fea0 0x7f99e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.692+0000 7f99f7fff640 1 -- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f815fea0 msgr2=0x7f99e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.692+0000 7f99f7fff640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f815fea0 0x7f99e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.692+0000 7f99f7fff640 1 -- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99e00039e0 msgr2=0x7f99e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.692+0000 7f99f7fff640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99e00039e0 0x7f99e0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.692+0000 7f99f7fff640 1 -- 172.21.15.7:0/357849628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e0003520 con 0x7f99e0004d40 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.692+0000 7f99f7fff640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99e00051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f99f8069940 tx=0x7f99e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b36801be1500bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.692+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f000a480 con 0x7f99e0004d40 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.692+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f002f070 con 0x7f99e0004d40 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.692+0000 7f99f67fc640 1 -- 172.21.15.7:0/357849628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f002bc70 con 0x7f99e0004d40 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.693+0000 7f99fdf6d640 1 -- 172.21.15.7:0/357849628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 msgr2=0x7f99e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:22.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.693+0000 7f99fdf6d640 1 --2- 172.21.15.7:0/357849628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99e00051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f99f8069940 tx=0x7f99e0005830 comp rx=0 tx=0).stop 2024-09-15T23:36:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.693+0000 7f99f7fff640 1 -- 172.21.15.7:0/357849628 reap_dead start 2024-09-15T23:36:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.693+0000 7f99fdf6d640 1 -- 172.21.15.7:0/357849628 shutdown_connections 2024-09-15T23:36:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.693+0000 7f99fdf6d640 1 -- 172.21.15.7:0/357849628 >> 172.21.15.7:0/357849628 conn(0x7f99f80b3f40 msgr2=0x7f99f80b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.693+0000 7f99fdf6d640 1 -- 172.21.15.7:0/357849628 shutdown_connections 2024-09-15T23:36:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.693+0000 7f99fdf6d640 1 -- 172.21.15.7:0/357849628 wait complete. 2024-09-15T23:36:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.694+0000 7f99fdf6d640 1 Processor -- start 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99fdf6d640 1 -- start start 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99fdf6d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99f81ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99fdf6d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f81ef9b0 0x7f99f80c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99fdf6d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99f80c8820 0x7f99f81f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99fdf6d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99f80b4a60 con 0x7f99e0004d40 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99fdf6d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99f80ce780 con 0x7f99f80c8820 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99fdf6d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f99f80b4be0 con 0x7f99f81ef9b0 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99f81ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99f81ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49716/0 (socket says 172.21.15.7:49716) 2024-09-15T23:36:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99f77fe640 1 -- 172.21.15.7:0/1983474533 learned_addr learned my addr 172.21.15.7:0/1983474533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99f6ffd640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f81ef9b0 0x7f99f80c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.695+0000 7f99f7fff640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99f80c8820 0x7f99f81f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99f77fe640 1 -- 172.21.15.7:0/1983474533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f81ef9b0 msgr2=0x7f99f80c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99f77fe640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f81ef9b0 0x7f99f80c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99f77fe640 1 -- 172.21.15.7:0/1983474533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99f80c8820 msgr2=0x7f99f81f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99f77fe640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99f80c8820 0x7f99f81f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99f77fe640 1 -- 172.21.15.7:0/1983474533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f80b5280 con 0x7f99e0004d40 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99f77fe640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99f81ef470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99e80119a0 tx=0x7f99e8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99dffff640 1 -- 172.21.15.7:0/1983474533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e800faf0 con 0x7f99e0004d40 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99dffff640 1 -- 172.21.15.7:0/1983474533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e800fc90 con 0x7f99e0004d40 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99dffff640 1 -- 172.21.15.7:0/1983474533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8014440 con 0x7f99e0004d40 2024-09-15T23:36:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f80b5560 con 0x7f99e0004d40 2024-09-15T23:36:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.696+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f80b5b20 con 0x7f99e0004d40 2024-09-15T23:36:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.697+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f80b3920 con 0x7f99e0004d40 2024-09-15T23:36:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.697+0000 7f99dffff640 1 -- 172.21.15.7:0/1983474533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f99e8012040 con 0x7f99e0004d40 2024-09-15T23:36:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.698+0000 7f99dffff640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f99e406c2b0 0x7f99e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.698+0000 7f99dffff640 1 -- 172.21.15.7:0/1983474533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f99e808b950 con 0x7f99e0004d40 2024-09-15T23:36:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.698+0000 7f99f6ffd640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f99e406c2b0 0x7f99e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.698+0000 7f99f6ffd640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f99e406c2b0 0x7f99e406e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f99f80cdd30 tx=0x7f99ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.703+0000 7f99dffff640 1 -- 172.21.15.7:0/1983474533 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f99e8019030 con 0x7f99e0004d40 2024-09-15T23:36:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:22.914+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f99f80ceed0 con 0x7f99e0004d40 2024-09-15T23:36:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.562+0000 7f99dffff640 1 -- 172.21.15.7:0/1983474533 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f99e8059ad0 con 0x7f99e0004d40 2024-09-15T23:36:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-09-15T23:36:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f99e406c2b0 msgr2=0x7f99e406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f99e406c2b0 0x7f99e406e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f99f80cdd30 tx=0x7f99ec009040 comp rx=0 tx=0).stop 2024-09-15T23:36:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 msgr2=0x7f99f81ef470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99f81ef470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99e80119a0 tx=0x7f99e8011e60 comp rx=0 tx=0).stop 2024-09-15T23:36:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 shutdown_connections 2024-09-15T23:36:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99f80c8820 0x7f99f81f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f99e406c2b0 0x7f99e406e770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99f81ef9b0 0x7f99f80c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 --2- 172.21.15.7:0/1983474533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0004d40 0x7f99f81ef470 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99e80119a0 tx=0x7f99e8011e60 comp rx=0 tx=0).stop 2024-09-15T23:36:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.569+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 >> 172.21.15.7:0/1983474533 conn(0x7f99f80b3f40 msgr2=0x7f99f80cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.574+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 shutdown_connections 2024-09-15T23:36:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.575+0000 7f99fdf6d640 1 -- 172.21.15.7:0/1983474533 wait complete. 2024-09-15T23:36:23.606 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-15T23:36:23.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-15T23:36:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.718+0000 7f6098dfd640 1 Processor -- start 2024-09-15T23:36:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.718+0000 7f6098dfd640 1 -- start start 2024-09-15T23:36:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.719+0000 7f6098dfd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406bc10 0x7f60940cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.719+0000 7f6098dfd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f609406fa90 0x7f60940aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.719+0000 7f6098dfd640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f609406f290 con 0x7f609406fef0 2024-09-15T23:36:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.719+0000 7f6098dfd640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f609406f570 con 0x7f609406fa90 2024-09-15T23:36:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.719+0000 7f6098dfd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f60940cb780 con 0x7f609406bc10 2024-09-15T23:36:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.720+0000 7f6091d9b640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f609406fa90 0x7f60940aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.720+0000 7f609259c640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406bc10 0x7f60940cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.720+0000 7f6091d9b640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f609406fa90 0x7f60940aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:60074/0 (socket says 172.21.15.7:60074) 2024-09-15T23:36:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.720+0000 7f6091d9b640 1 -- 172.21.15.7:0/2252680650 learned_addr learned my addr 172.21.15.7:0/2252680650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.720+0000 7f6091d9b640 1 -- 172.21.15.7:0/2252680650 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406bc10 msgr2=0x7f60940cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.720+0000 7f6091d9b640 1 --2- 172.21.15.7:0/2252680650 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406bc10 0x7f60940cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.720+0000 7f6091d9b640 1 -- 172.21.15.7:0/2252680650 >> v1:172.21.15.18:6789/0 conn(0x7f609406fef0 legacy=0x7f609406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.720+0000 7f6091d9b640 1 -- 172.21.15.7:0/2252680650 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60940cb4a0 con 0x7f609406fa90 2024-09-15T23:36:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.720+0000 7f609159a640 1 -- 172.21.15.7:0/2252680650 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120954786 0 0) 0x7f609406f290 con 0x7f609406fef0 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.721+0000 7f6091d9b640 1 --2- 172.21.15.7:0/2252680650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406fa90 0x7f60940aec60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f607c0098a0 tx=0x7f607c02a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=220f256798856ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.721+0000 7f609159a640 1 -- 172.21.15.7:0/2252680650 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c037070 con 0x7f609406fa90 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.721+0000 7f609159a640 1 -- 172.21.15.7:0/2252680650 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607c032440 con 0x7f609406fa90 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.721+0000 7f609159a640 1 -- 172.21.15.7:0/2252680650 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c032810 con 0x7f609406fa90 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.722+0000 7f6098dfd640 1 -- 172.21.15.7:0/2252680650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406fa90 msgr2=0x7f60940aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.722+0000 7f6098dfd640 1 --2- 172.21.15.7:0/2252680650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406fa90 0x7f60940aec60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f607c0098a0 tx=0x7f607c02a290 comp rx=0 tx=0).stop 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.722+0000 7f6098dfd640 1 -- 172.21.15.7:0/2252680650 shutdown_connections 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.722+0000 7f6098dfd640 1 --2- 172.21.15.7:0/2252680650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406fa90 0x7f60940aec60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.722+0000 7f6098dfd640 1 --2- 172.21.15.7:0/2252680650 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406bc10 0x7f60940cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.722+0000 7f6098dfd640 1 -- 172.21.15.7:0/2252680650 >> 172.21.15.7:0/2252680650 conn(0x7f60940b2fc0 msgr2=0x7f60940b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.722+0000 7f6098dfd640 1 -- 172.21.15.7:0/2252680650 shutdown_connections 2024-09-15T23:36:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.722+0000 7f6098dfd640 1 -- 172.21.15.7:0/2252680650 wait complete. 2024-09-15T23:36:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.723+0000 7f6098dfd640 1 Processor -- start 2024-09-15T23:36:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.723+0000 7f6098dfd640 1 -- start start 2024-09-15T23:36:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.724+0000 7f6098dfd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406bc10 0x7f60941584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.724+0000 7f6098dfd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406fa90 0x7f609415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.724+0000 7f6098dfd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609406fef0 0x7f609415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.724+0000 7f6098dfd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f609406c880 con 0x7f609406fef0 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.724+0000 7f6098dfd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f609406c700 con 0x7f609406bc10 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.724+0000 7f6098dfd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f609406ca00 con 0x7f609406fa90 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.724+0000 7f6092d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609406fef0 0x7f609415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.724+0000 7f6092d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609406fef0 0x7f609415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49728/0 (socket says 172.21.15.7:49728) 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.724+0000 7f6092d9d640 1 -- 172.21.15.7:0/2793713666 learned_addr learned my addr 172.21.15.7:0/2793713666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f609259c640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406bc10 0x7f60941584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6091d9b640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406fa90 0x7f609415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6092d9d640 1 -- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406fa90 msgr2=0x7f609415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6092d9d640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406fa90 0x7f609415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6092d9d640 1 -- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406bc10 msgr2=0x7f60941584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6092d9d640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406bc10 0x7f60941584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6092d9d640 1 -- 172.21.15.7:0/2793713666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60941f4460 con 0x7f609406fef0 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f609259c640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406bc10 0x7f60941584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6091d9b640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406fa90 0x7f609415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6092d9d640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609406fef0 0x7f609415c6a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f608400c410 tx=0x7f608400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f608affd640 1 -- 172.21.15.7:0/2793713666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6084017070 con 0x7f609406fef0 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f608affd640 1 -- 172.21.15.7:0/2793713666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60840035f0 con 0x7f609406fef0 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60941f4720 con 0x7f609406fef0 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f608affd640 1 -- 172.21.15.7:0/2793713666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60840039c0 con 0x7f609406fef0 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.725+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60941f4ce0 con 0x7f609406fef0 2024-09-15T23:36:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.726+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60940cadb0 con 0x7f609406fef0 2024-09-15T23:36:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.727+0000 7f608affd640 1 -- 172.21.15.7:0/2793713666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f608401a070 con 0x7f609406fef0 2024-09-15T23:36:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.727+0000 7f608affd640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f607406c2e0 0x7f607406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.727+0000 7f608affd640 1 -- 172.21.15.7:0/2793713666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f608408a740 con 0x7f609406fef0 2024-09-15T23:36:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.727+0000 7f609259c640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f607406c2e0 0x7f607406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.728+0000 7f609259c640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f607406c2e0 0x7f607406e7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6094067dd0 tx=0x7f608c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.732+0000 7f608affd640 1 -- 172.21.15.7:0/2793713666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6084058eb0 con 0x7f609406fef0 2024-09-15T23:36:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:23.914+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f60940d0fa0 con 0x7f609406fef0 2024-09-15T23:36:24.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.574+0000 7f608affd640 1 -- 172.21.15.7:0/2793713666 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f60840587b0 con 0x7f609406fef0 2024-09-15T23:36:24.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f607406c2e0 msgr2=0x7f607406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:24.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f607406c2e0 0x7f607406e7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6094067dd0 tx=0x7f608c00b040 comp rx=0 tx=0).stop 2024-09-15T23:36:24.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609406fef0 msgr2=0x7f609415c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:24.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609406fef0 0x7f609415c6a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f608400c410 tx=0x7f608400c8d0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 shutdown_connections 2024-09-15T23:36:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609406fef0 0x7f609415c6a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f607406c2e0 0x7f607406e7a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f609406fa90 0x7f609415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 --2- 172.21.15.7:0/2793713666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f609406bc10 0x7f60941584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 >> 172.21.15.7:0/2793713666 conn(0x7f60940b2fc0 msgr2=0x7f609415d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 shutdown_connections 2024-09-15T23:36:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.578+0000 7f6098dfd640 1 -- 172.21.15.7:0/2793713666 wait complete. 2024-09-15T23:36:24.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-15T23:36:24.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.710+0000 7f66f599f640 1 Processor -- start 2024-09-15T23:36:24.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.710+0000 7f66f599f640 1 -- start start 2024-09-15T23:36:24.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66f599f640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66f00b3590 0x7f66f00d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:24.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66f599f640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f66f00d2f20 0x7f66f006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66f599f640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66f0161df0 con 0x7f66f0161a10 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66f599f640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f66f00d3370 con 0x7f66f00b3590 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66f599f640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f66f00d0d90 con 0x7f66f00d2f20 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66effff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66f00b3590 0x7f66f00d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66effff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66f00b3590 0x7f66f00d29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49744/0 (socket says 172.21.15.7:49744) 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66effff640 1 -- 172.21.15.7:0/4210016889 learned_addr learned my addr 172.21.15.7:0/4210016889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66f519e640 1 --2- 172.21.15.7:0/4210016889 >> v2:172.21.15.18:3300/0 conn(0x7f66f00d2f20 0x7f66f006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66effff640 1 -- 172.21.15.7:0/4210016889 >> v2:172.21.15.18:3300/0 conn(0x7f66f00d2f20 msgr2=0x7f66f006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66effff640 1 --2- 172.21.15.7:0/4210016889 >> v2:172.21.15.18:3300/0 conn(0x7f66f00d2f20 0x7f66f006cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66effff640 1 -- 172.21.15.7:0/4210016889 >> v1:172.21.15.18:6789/0 conn(0x7f66f0161a10 legacy=0x7f66f00d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.711+0000 7f66effff640 1 -- 172.21.15.7:0/4210016889 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f00d1070 con 0x7f66f00b3590 2024-09-15T23:36:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.712+0000 7f66effff640 1 --2- 172.21.15.7:0/4210016889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 0x7f66f00d29e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66f00ceea0 tx=0x7f66dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4fa8251017f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.712+0000 7f66ef7fe640 1 -- 172.21.15.7:0/4210016889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc037070 con 0x7f66f00b3590 2024-09-15T23:36:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.712+0000 7f66ef7fe640 1 -- 172.21.15.7:0/4210016889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66dc032440 con 0x7f66f00b3590 2024-09-15T23:36:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.712+0000 7f66ef7fe640 1 -- 172.21.15.7:0/4210016889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc032810 con 0x7f66f00b3590 2024-09-15T23:36:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.713+0000 7f66f599f640 1 -- 172.21.15.7:0/4210016889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 msgr2=0x7f66f00d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.713+0000 7f66f599f640 1 --2- 172.21.15.7:0/4210016889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 0x7f66f00d29e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66f00ceea0 tx=0x7f66dc02a230 comp rx=0 tx=0).stop 2024-09-15T23:36:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.713+0000 7f66f599f640 1 -- 172.21.15.7:0/4210016889 shutdown_connections 2024-09-15T23:36:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.713+0000 7f66f599f640 1 --2- 172.21.15.7:0/4210016889 >> v2:172.21.15.18:3300/0 conn(0x7f66f00d2f20 0x7f66f006cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.713+0000 7f66f599f640 1 --2- 172.21.15.7:0/4210016889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 0x7f66f00d29e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.713+0000 7f66f599f640 1 -- 172.21.15.7:0/4210016889 >> 172.21.15.7:0/4210016889 conn(0x7f66f009d7c0 msgr2=0x7f66f00d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.713+0000 7f66f599f640 1 -- 172.21.15.7:0/4210016889 shutdown_connections 2024-09-15T23:36:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.713+0000 7f66f599f640 1 -- 172.21.15.7:0/4210016889 wait complete. 2024-09-15T23:36:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.714+0000 7f66f599f640 1 Processor -- start 2024-09-15T23:36:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.714+0000 7f66f599f640 1 -- start start 2024-09-15T23:36:24.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f599f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 0x7f66f01519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:24.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f599f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f00d2f20 0x7f66f015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f599f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66f0161a10 0x7f66f015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f599f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66f006d320 con 0x7f66f00b3590 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f599f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66f006d1a0 con 0x7f66f00d2f20 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f599f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f66f006d4a0 con 0x7f66f0161a10 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f499d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 0x7f66f01519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f499d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 0x7f66f01519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49758/0 (socket says 172.21.15.7:49758) 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f499d640 1 -- 172.21.15.7:0/3896681805 learned_addr learned my addr 172.21.15.7:0/3896681805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66effff640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f00d2f20 0x7f66f015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f519e640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66f0161a10 0x7f66f015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f499d640 1 -- 172.21.15.7:0/3896681805 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66f0161a10 msgr2=0x7f66f015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f499d640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66f0161a10 0x7f66f015ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f499d640 1 -- 172.21.15.7:0/3896681805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f00d2f20 msgr2=0x7f66f015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f499d640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f00d2f20 0x7f66f015b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.715+0000 7f66f499d640 1 -- 172.21.15.7:0/3896681805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f00b9230 con 0x7f66f00b3590 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.716+0000 7f66f499d640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 0x7f66f01519a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66e400b3f0 tx=0x7f66e400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.716+0000 7f66ed7fa640 1 -- 172.21.15.7:0/3896681805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4015020 con 0x7f66f00b3590 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.716+0000 7f66ed7fa640 1 -- 172.21.15.7:0/3896681805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e4011830 con 0x7f66f00b3590 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.716+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f00b9510 con 0x7f66f00b3590 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.716+0000 7f66ed7fa640 1 -- 172.21.15.7:0/3896681805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011c20 con 0x7f66f00b3590 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.716+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f00b9ad0 con 0x7f66f00b3590 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.717+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f00d29e0 con 0x7f66f00b3590 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.717+0000 7f66ed7fa640 1 -- 172.21.15.7:0/3896681805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f66e4018020 con 0x7f66f00b3590 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.718+0000 7f66ed7fa640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f66d406c2b0 0x7f66d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.718+0000 7f66ed7fa640 1 -- 172.21.15.7:0/3896681805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f66e408afd0 con 0x7f66f00b3590 2024-09-15T23:36:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.718+0000 7f66effff640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f66d406c2b0 0x7f66d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:24.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.718+0000 7f66effff640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f66d406c2b0 0x7f66d406e770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66dc034000 tx=0x7f66dc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:24.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.722+0000 7f66ed7fa640 1 -- 172.21.15.7:0/3896681805 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66e401f030 con 0x7f66f00b3590 2024-09-15T23:36:24.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:24.899+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f66f00d0680 con 0x7f66f00b3590 2024-09-15T23:36:25.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:36:25.500 DEBUG:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:36:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.578+0000 7f66ed7fa640 1 -- 172.21.15.7:0/3896681805 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f66e4059050 con 0x7f66f00b3590 2024-09-15T23:36:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' created 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.582+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f66d406c2b0 msgr2=0x7f66d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.582+0000 7f66f599f640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f66d406c2b0 0x7f66d406e770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66dc034000 tx=0x7f66dc02e000 comp rx=0 tx=0).stop 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.582+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 msgr2=0x7f66f01519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.582+0000 7f66f599f640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 0x7f66f01519a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66e400b3f0 tx=0x7f66e400b8b0 comp rx=0 tx=0).stop 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.583+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 shutdown_connections 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.583+0000 7f66f599f640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f66d406c2b0 0x7f66d406e770 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66dc034000 tx=0x7f66dc02e000 comp rx=0 tx=0).stop 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.583+0000 7f66f599f640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66f0161a10 0x7f66f015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.583+0000 7f66f599f640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f00d2f20 0x7f66f015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.583+0000 7f66f599f640 1 --2- 172.21.15.7:0/3896681805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f00b3590 0x7f66f01519a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.583+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 >> 172.21.15.7:0/3896681805 conn(0x7f66f009d7c0 msgr2=0x7f66f00b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.586+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 shutdown_connections 2024-09-15T23:36:26.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.587+0000 7f66f599f640 1 -- 172.21.15.7:0/3896681805 wait complete. 2024-09-15T23:36:26.604 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-15T23:36:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.715+0000 7fa120ca5640 1 Processor -- start 2024-09-15T23:36:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.715+0000 7fa120ca5640 1 -- start start 2024-09-15T23:36:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa120ca5640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa11c159e30 0x7fa11c06dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa120ca5640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c0ad850 0x7fa11c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa120ca5640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa11c06e480 con 0x7fa11c06f520 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa120ca5640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa11c06e760 con 0x7fa11c159e30 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa120ca5640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c06ea40 con 0x7fa11c0ad850 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa11a59c640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa11c159e30 0x7fa11c06dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa11ad9d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c0ad850 0x7fa11c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa119d9b640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fa11c06f520 0x7fa11c0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:42640/0 (socket says 172.21.15.7:42640) 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa11ad9d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c0ad850 0x7fa11c15d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:47850/0 (socket says 172.21.15.7:47850) 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa11a59c640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa11c159e30 0x7fa11c06dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:60102/0 (socket says 172.21.15.7:60102) 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.716+0000 7fa119d9b640 1 -- 172.21.15.7:0/1215633544 learned_addr learned my addr 172.21.15.7:0/1215633544 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.717+0000 7fa11a59c640 1 -- 172.21.15.7:0/1215633544 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c0ad850 msgr2=0x7fa11c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.717+0000 7fa11a59c640 1 --2- 172.21.15.7:0/1215633544 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c0ad850 0x7fa11c15d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:26.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.717+0000 7fa11a59c640 1 -- 172.21.15.7:0/1215633544 >> v1:172.21.15.18:6789/0 conn(0x7fa11c06f520 legacy=0x7fa11c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:26.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.717+0000 7fa11a59c640 1 -- 172.21.15.7:0/1215633544 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c06ed20 con 0x7fa11c159e30 2024-09-15T23:36:26.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.717+0000 7fa11959a640 1 -- 172.21.15.7:0/1215633544 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672594910 0 0) 0x7fa11c06e480 con 0x7fa11c06f520 2024-09-15T23:36:26.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.717+0000 7fa11a59c640 1 --2- 172.21.15.7:0/1215633544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa11c159e30 0x7fa11c06dea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0097a0 tx=0x7fa10c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f5cc49949900c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:26.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa11959a640 1 -- 172.21.15.7:0/1215633544 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c037070 con 0x7fa11c159e30 2024-09-15T23:36:26.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa11959a640 1 -- 172.21.15.7:0/1215633544 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10c032440 con 0x7fa11c159e30 2024-09-15T23:36:26.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa11959a640 1 -- 172.21.15.7:0/1215633544 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c032810 con 0x7fa11c159e30 2024-09-15T23:36:26.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa120ca5640 1 -- 172.21.15.7:0/1215633544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa11c159e30 msgr2=0x7fa11c06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa120ca5640 1 --2- 172.21.15.7:0/1215633544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa11c159e30 0x7fa11c06dea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0097a0 tx=0x7fa10c02a180 comp rx=0 tx=0).stop 2024-09-15T23:36:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa120ca5640 1 -- 172.21.15.7:0/1215633544 shutdown_connections 2024-09-15T23:36:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa120ca5640 1 --2- 172.21.15.7:0/1215633544 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c0ad850 0x7fa11c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa120ca5640 1 --2- 172.21.15.7:0/1215633544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa11c159e30 0x7fa11c06dea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa120ca5640 1 -- 172.21.15.7:0/1215633544 >> 172.21.15.7:0/1215633544 conn(0x7fa11c0b1e60 msgr2=0x7fa11c0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa120ca5640 1 -- 172.21.15.7:0/1215633544 shutdown_connections 2024-09-15T23:36:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.718+0000 7fa120ca5640 1 -- 172.21.15.7:0/1215633544 wait complete. 2024-09-15T23:36:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.720+0000 7fa120ca5640 1 Processor -- start 2024-09-15T23:36:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.720+0000 7fa120ca5640 1 -- start start 2024-09-15T23:36:26.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.720+0000 7fa120ca5640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c06f520 0x7fa11c099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:26.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.720+0000 7fa120ca5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11c0ad850 0x7fa11c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:26.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.720+0000 7fa120ca5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa11c159e30 0x7fa11c0aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:26.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.720+0000 7fa120ca5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c15a390 con 0x7fa11c0ad850 2024-09-15T23:36:26.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.720+0000 7fa120ca5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c15a210 con 0x7fa11c159e30 2024-09-15T23:36:26.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.720+0000 7fa120ca5640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c0b3160 con 0x7fa11c06f520 2024-09-15T23:36:26.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa119d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11c0ad850 0x7fa11c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:26.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa119d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11c0ad850 0x7fa11c157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49766/0 (socket says 172.21.15.7:49766) 2024-09-15T23:36:26.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa119d9b640 1 -- 172.21.15.7:0/2949679510 learned_addr learned my addr 172.21.15.7:0/2949679510 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:26.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa11ad9d640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa11c159e30 0x7fa11c0aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:26.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa11a59c640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c06f520 0x7fa11c099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:26.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa119d9b640 1 -- 172.21.15.7:0/2949679510 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c06f520 msgr2=0x7fa11c099520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:26.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa119d9b640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c06f520 0x7fa11c099520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:26.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa119d9b640 1 -- 172.21.15.7:0/2949679510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa11c159e30 msgr2=0x7fa11c0aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:26.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa119d9b640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa11c159e30 0x7fa11c0aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:26.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa119d9b640 1 -- 172.21.15.7:0/2949679510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c156240 con 0x7fa11c0ad850 2024-09-15T23:36:26.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa11a59c640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c06f520 0x7fa11c099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:26.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa119d9b640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11c0ad850 0x7fa11c157c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c4d0 tx=0x7fa11000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:26.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa10affd640 1 -- 172.21.15.7:0/2949679510 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110018070 con 0x7fa11c0ad850 2024-09-15T23:36:26.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa10affd640 1 -- 172.21.15.7:0/2949679510 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa110007500 con 0x7fa11c0ad850 2024-09-15T23:36:26.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c156520 con 0x7fa11c0ad850 2024-09-15T23:36:26.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa10affd640 1 -- 172.21.15.7:0/2949679510 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110004a20 con 0x7fa11c0ad850 2024-09-15T23:36:26.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.721+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11c156ae0 con 0x7fa11c0ad850 2024-09-15T23:36:26.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.723+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11c068480 con 0x7fa11c0ad850 2024-09-15T23:36:26.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.723+0000 7fa10affd640 1 -- 172.21.15.7:0/2949679510 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa11001b070 con 0x7fa11c0ad850 2024-09-15T23:36:26.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.723+0000 7fa10affd640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa0fc06c2b0 0x7fa0fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:26.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.724+0000 7fa11a59c640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa0fc06c2b0 0x7fa0fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:26.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.724+0000 7fa10affd640 1 -- 172.21.15.7:0/2949679510 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fa11008a970 con 0x7fa11c0ad850 2024-09-15T23:36:26.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.724+0000 7fa11a59c640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa0fc06c2b0 0x7fa0fc06e770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa10c034000 tx=0x7fa10c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:26.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.728+0000 7fa10affd640 1 -- 172.21.15.7:0/2949679510 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa11c068480 con 0x7fa11c0ad850 2024-09-15T23:36:26.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:26.902+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fa11c06fc80 con 0x7fa11c0ad850 2024-09-15T23:36:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.580+0000 7fa10affd640 1 -- 172.21.15.7:0/2949679510 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7fa110058800 con 0x7fa11c0ad850 2024-09-15T23:36:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:set pool 4 allow_ec_overwrites to true 2024-09-15T23:36:27.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa0fc06c2b0 msgr2=0x7fa0fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:27.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa0fc06c2b0 0x7fa0fc06e770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa10c034000 tx=0x7fa10c02e000 comp rx=0 tx=0).stop 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11c0ad850 msgr2=0x7fa11c157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11c0ad850 0x7fa11c157c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c4d0 tx=0x7fa11000c990 comp rx=0 tx=0).stop 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 shutdown_connections 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa11c159e30 0x7fa11c0aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11c0ad850 0x7fa11c157c20 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c4d0 tx=0x7fa11000c990 comp rx=0 tx=0).stop 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa0fc06c2b0 0x7fa0fc06e770 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 --2- 172.21.15.7:0/2949679510 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa11c06f520 0x7fa11c099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 >> 172.21.15.7:0/2949679510 conn(0x7fa11c0b1e60 msgr2=0x7fa11c0b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 shutdown_connections 2024-09-15T23:36:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.586+0000 7fa120ca5640 1 -- 172.21.15.7:0/2949679510 wait complete. 2024-09-15T23:36:27.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-15T23:36:27.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.727+0000 7f06cde0e640 1 Processor -- start 2024-09-15T23:36:27.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.727+0000 7f06cde0e640 1 -- start start 2024-09-15T23:36:27.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.728+0000 7f06cde0e640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f06c815e5b0 0x7f06c8162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:27.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.728+0000 7f06cde0e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06c815e990 0x7f06c80cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:27.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.728+0000 7f06cde0e640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c80d2980 con 0x7f06c80d24e0 2024-09-15T23:36:27.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.728+0000 7f06cde0e640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f06c80d2c40 con 0x7f06c815e990 2024-09-15T23:36:27.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.728+0000 7f06cde0e640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f06c80d2f20 con 0x7f06c815e5b0 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.728+0000 7f06c7fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06c815e990 0x7f06c80cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.728+0000 7f06c7fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06c815e990 0x7f06c80cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53558/0 (socket says 172.21.15.7:53558) 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.728+0000 7f06c7fff640 1 -- 172.21.15.7:0/251557645 learned_addr learned my addr 172.21.15.7:0/251557645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.728+0000 7f06c77fe640 1 --2- 172.21.15.7:0/251557645 >> v2:172.21.15.18:3300/0 conn(0x7f06c815e5b0 0x7f06c8162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.729+0000 7f06c7fff640 1 -- 172.21.15.7:0/251557645 >> v2:172.21.15.18:3300/0 conn(0x7f06c815e5b0 msgr2=0x7f06c8162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.729+0000 7f06c7fff640 1 --2- 172.21.15.7:0/251557645 >> v2:172.21.15.18:3300/0 conn(0x7f06c815e5b0 0x7f06c8162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.729+0000 7f06c7fff640 1 -- 172.21.15.7:0/251557645 >> v1:172.21.15.18:6789/0 conn(0x7f06c80d24e0 legacy=0x7f06c8162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.729+0000 7f06c7fff640 1 -- 172.21.15.7:0/251557645 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c80d3200 con 0x7f06c815e990 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.729+0000 7f06c77fe640 1 --2- 172.21.15.7:0/251557645 >> v2:172.21.15.18:3300/0 conn(0x7f06c815e5b0 0x7f06c8162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.729+0000 7f06c7fff640 1 --2- 172.21.15.7:0/251557645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 0x7f06c80cf420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009e50 tx=0x7f06c002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c422f40696582e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.729+0000 7f06c67fc640 1 -- 172.21.15.7:0/251557645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0036070 con 0x7f06c815e990 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06c67fc640 1 -- 172.21.15.7:0/251557645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c0031600 con 0x7f06c815e990 2024-09-15T23:36:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06c67fc640 1 -- 172.21.15.7:0/251557645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00319d0 con 0x7f06c815e990 2024-09-15T23:36:27.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06cde0e640 1 -- 172.21.15.7:0/251557645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 msgr2=0x7f06c80cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06cde0e640 1 --2- 172.21.15.7:0/251557645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 0x7f06c80cf420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009e50 tx=0x7f06c002a760 comp rx=0 tx=0).stop 2024-09-15T23:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06cde0e640 1 -- 172.21.15.7:0/251557645 shutdown_connections 2024-09-15T23:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06cde0e640 1 --2- 172.21.15.7:0/251557645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 0x7f06c80cf420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06cde0e640 1 --2- 172.21.15.7:0/251557645 >> v2:172.21.15.18:3300/0 conn(0x7f06c815e5b0 0x7f06c8162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06cde0e640 1 -- 172.21.15.7:0/251557645 >> 172.21.15.7:0/251557645 conn(0x7f06c808bfd0 msgr2=0x7f06c80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06cde0e640 1 -- 172.21.15.7:0/251557645 shutdown_connections 2024-09-15T23:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.730+0000 7f06cde0e640 1 -- 172.21.15.7:0/251557645 wait complete. 2024-09-15T23:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.731+0000 7f06cde0e640 1 Processor -- start 2024-09-15T23:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06cde0e640 1 -- start start 2024-09-15T23:36:27.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06cde0e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06c80d24e0 0x7f06c80c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:27.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06cde0e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c815e5b0 0x7f06c80b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:27.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06cde0e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 0x7f06c80c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:27.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06cde0e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06c80b3b40 con 0x7f06c815e990 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06cde0e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f06c80b39c0 con 0x7f06c815e5b0 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06cde0e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f06c8162fa0 con 0x7f06c80d24e0 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 0x7f06c80c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 0x7f06c80c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53568/0 (socket says 172.21.15.7:53568) 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06c7fff640 1 -- 172.21.15.7:0/1241201756 learned_addr learned my addr 172.21.15.7:0/1241201756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06c77fe640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06c80d24e0 0x7f06c80c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.732+0000 7f06c6ffd640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c815e5b0 0x7f06c80b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06c7fff640 1 -- 172.21.15.7:0/1241201756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06c80d24e0 msgr2=0x7f06c80c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06c7fff640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06c80d24e0 0x7f06c80c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06c7fff640 1 -- 172.21.15.7:0/1241201756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c815e5b0 msgr2=0x7f06c80b81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06c7fff640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c815e5b0 0x7f06c80b81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06c7fff640 1 -- 172.21.15.7:0/1241201756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c80b86f0 con 0x7f06c815e990 2024-09-15T23:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06c7fff640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 0x7f06c80c80d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06c0034040 tx=0x7f06c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06affff640 1 -- 172.21.15.7:0/1241201756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0036050 con 0x7f06c815e990 2024-09-15T23:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06affff640 1 -- 172.21.15.7:0/1241201756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c0042070 con 0x7f06c815e990 2024-09-15T23:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c80b89a0 con 0x7f06c815e990 2024-09-15T23:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06affff640 1 -- 172.21.15.7:0/1241201756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0009510 con 0x7f06c815e990 2024-09-15T23:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.733+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c80b8f60 con 0x7f06c815e990 2024-09-15T23:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.734+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c8162350 con 0x7f06c815e990 2024-09-15T23:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.735+0000 7f06affff640 1 -- 172.21.15.7:0/1241201756 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f06c0007bd0 con 0x7f06c815e990 2024-09-15T23:36:27.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.735+0000 7f06affff640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f06b406c2b0 0x7f06b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:27.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.735+0000 7f06affff640 1 -- 172.21.15.7:0/1241201756 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f06c00ac2f0 con 0x7f06c815e990 2024-09-15T23:36:27.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.735+0000 7f06c77fe640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f06b406c2b0 0x7f06b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:27.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.735+0000 7f06c77fe640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f06b406c2b0 0x7f06b406e770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006e80 tx=0x7f06b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:27.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.739+0000 7f06affff640 1 -- 172.21.15.7:0/1241201756 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f06c007a180 con 0x7f06c815e990 2024-09-15T23:36:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:27.957+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f06c80ba560 con 0x7f06c815e990 2024-09-15T23:36:28.541 INFO:tasks.ceph.mds.d.smithi018.stderr:2024-09-15T23:36:28.540+0000 7f9d546c2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T23:36:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.624+0000 7f06affff640 1 -- 172.21.15.7:0/1241201756 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7f06c007e480 con 0x7f06c815e990 2024-09-15T23:36:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:added data pool 4 to fsmap 2024-09-15T23:36:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.627+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f06b406c2b0 msgr2=0x7f06b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.627+0000 7f06cde0e640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f06b406c2b0 0x7f06b406e770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006e80 tx=0x7f06b8008040 comp rx=0 tx=0).stop 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.627+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 msgr2=0x7f06c80c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.627+0000 7f06cde0e640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 0x7f06c80c80d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06c0034040 tx=0x7f06c0004a40 comp rx=0 tx=0).stop 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.627+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 shutdown_connections 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.628+0000 7f06cde0e640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c815e990 0x7f06c80c80d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.628+0000 7f06cde0e640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c815e5b0 0x7f06c80b81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.628+0000 7f06cde0e640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f06b406c2b0 0x7f06b406e770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.628+0000 7f06cde0e640 1 --2- 172.21.15.7:0/1241201756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06c80d24e0 0x7f06c80c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.628+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 >> 172.21.15.7:0/1241201756 conn(0x7f06c808bfd0 msgr2=0x7f06c815c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.628+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 shutdown_connections 2024-09-15T23:36:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.628+0000 7f06cde0e640 1 -- 172.21.15.7:0/1241201756 wait complete. 2024-09-15T23:36:28.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:36:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.749+0000 7f56fc6e4640 1 Processor -- start 2024-09-15T23:36:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.749+0000 7f56fc6e4640 1 -- start start 2024-09-15T23:36:28.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.750+0000 7f56fc6e4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f56f40d32a0 0x7f56f4169280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.750+0000 7f56fc6e4640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f40b3d10 con 0x7f56f415e6d0 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.750+0000 7f56fc6e4640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f40cfcd0 con 0x7f56f40d51c0 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.750+0000 7f56fc6e4640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f56f40d28c0 con 0x7f56f40d32a0 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.750+0000 7f56fa480640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f56f415e6d0 0x7f56f40d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44690/0 (socket says 172.21.15.7:44690) 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.750+0000 7f56fa480640 1 -- 172.21.15.7:0/2596043566 learned_addr learned my addr 172.21.15.7:0/2596043566 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.750+0000 7f56fac81640 1 --2- 172.21.15.7:0/2596043566 >> v2:172.21.15.18:3300/0 conn(0x7f56f40d32a0 0x7f56f4169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.750+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2529202748 0 0) 0x7f56f40b3d10 con 0x7f56f415e6d0 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.750+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56dc003520 con 0x7f56f415e6d0 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 214727423 0 0) 0x7f56dc003520 con 0x7f56f415e6d0 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 >> v2:172.21.15.18:3300/0 conn(0x7f56f40d32a0 msgr2=0x7f56f4169280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 --2- 172.21.15.7:0/2596043566 >> v2:172.21.15.18:3300/0 conn(0x7f56f40d32a0 0x7f56f4169280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 >> v1:172.21.15.18:6789/0 conn(0x7f56f40d51c0 legacy=0x7f56f40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:28.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f40cff50 con 0x7f56f415e6d0 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f56e40019f0 con 0x7f56f415e6d0 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 >> v1:172.21.15.7:6789/0 conn(0x7f56f415e6d0 legacy=0x7f56f40d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56dc0039b0 0x7f56dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc0043b0 0x7f56dc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56dc005080 0x7f56dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f56f40d28c0 con 0x7f56dc0043b0 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc00bfe0 con 0x7f56dc0039b0 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc00c1e0 con 0x7f56dc005080 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f9c7f640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc0043b0 0x7f56dc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56fa480640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56dc0039b0 0x7f56dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56fac81640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56dc005080 0x7f56dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f9c7f640 1 -- 172.21.15.7:0/2596043566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56dc005080 msgr2=0x7f56dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:28.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f9c7f640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56dc005080 0x7f56dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f9c7f640 1 -- 172.21.15.7:0/2596043566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56dc0039b0 msgr2=0x7f56dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f9c7f640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56dc0039b0 0x7f56dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56fac81640 1 -- 172.21.15.7:0/2596043566 reap_dead start 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.751+0000 7f56f9c7f640 1 -- 172.21.15.7:0/2596043566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc003520 con 0x7f56dc0043b0 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.752+0000 7f56f9c7f640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc0043b0 0x7f56dc004880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56e8001a00 tx=0x7f56dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae98e0e3b3482c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.752+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8037070 con 0x7f56dc0043b0 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.752+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e802ad80 con 0x7f56dc0043b0 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.752+0000 7f56f947e640 1 -- 172.21.15.7:0/2596043566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8001600 con 0x7f56dc0043b0 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.753+0000 7f56fc6e4640 1 -- 172.21.15.7:0/2596043566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc0043b0 msgr2=0x7f56dc004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.753+0000 7f56fc6e4640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc0043b0 0x7f56dc004880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56e8001a00 tx=0x7f56dc005830 comp rx=0 tx=0).stop 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.753+0000 7f56fc6e4640 1 -- 172.21.15.7:0/2596043566 shutdown_connections 2024-09-15T23:36:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.753+0000 7f56fc6e4640 1 --2- 172.21.15.7:0/2596043566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc0043b0 0x7f56dc004880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.753+0000 7f56fc6e4640 1 -- 172.21.15.7:0/2596043566 >> 172.21.15.7:0/2596043566 conn(0x7f56f408bfd0 msgr2=0x7f56f40d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:28.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.753+0000 7f56fc6e4640 1 -- 172.21.15.7:0/2596043566 shutdown_connections 2024-09-15T23:36:28.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.753+0000 7f56fc6e4640 1 -- 172.21.15.7:0/2596043566 wait complete. 2024-09-15T23:36:28.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.754+0000 7f56fc6e4640 1 Processor -- start 2024-09-15T23:36:28.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.754+0000 7f56fc6e4640 1 -- start start 2024-09-15T23:36:28.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fc6e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56dc0043b0 0x7f56f406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fc6e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f40ca430 0x7f56f40ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fc6e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56f41564e0 0x7f56f4156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fc6e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f56f40d0510 con 0x7f56f41564e0 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fc6e4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f56f415eab0 con 0x7f56f40ca430 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fc6e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f56f40d2b90 con 0x7f56dc0043b0 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fac81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56f41564e0 0x7f56f4156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fac81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56f41564e0 0x7f56f4156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53580/0 (socket says 172.21.15.7:53580) 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fac81640 1 -- 172.21.15.7:0/3358062669 learned_addr learned my addr 172.21.15.7:0/3358062669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fa480640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56dc0043b0 0x7f56f406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56f9c7f640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f40ca430 0x7f56f40ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fac81640 1 -- 172.21.15.7:0/3358062669 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56dc0043b0 msgr2=0x7f56f406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:28.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fac81640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56dc0043b0 0x7f56f406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fac81640 1 -- 172.21.15.7:0/3358062669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f40ca430 msgr2=0x7f56f40ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fac81640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f40ca430 0x7f56f40ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fac81640 1 -- 172.21.15.7:0/3358062669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f415ee80 con 0x7f56f41564e0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.755+0000 7f56fac81640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56f41564e0 0x7f56f4156940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56f40d43b0 tx=0x7f56ec011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.756+0000 7f56f2ffd640 1 -- 172.21.15.7:0/3358062669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec004030 con 0x7f56f41564e0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.756+0000 7f56f2ffd640 1 -- 172.21.15.7:0/3358062669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec01b070 con 0x7f56f41564e0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.756+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f415f160 con 0x7f56f41564e0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.756+0000 7f56f2ffd640 1 -- 172.21.15.7:0/3358062669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0043f0 con 0x7f56f41564e0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.756+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f415f720 con 0x7f56f41564e0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.757+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f40d2f20 con 0x7f56f41564e0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.757+0000 7f56f2ffd640 1 -- 172.21.15.7:0/3358062669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f56ec012040 con 0x7f56f41564e0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.758+0000 7f56f2ffd640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f56e006c2b0 0x7f56e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.758+0000 7f56fa480640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f56e006c2b0 0x7f56e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:28.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.758+0000 7f56f2ffd640 1 -- 172.21.15.7:0/3358062669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f56ec08b6b0 con 0x7f56f41564e0 2024-09-15T23:36:28.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.758+0000 7f56fa480640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f56e006c2b0 0x7f56e006e770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f56e4001560 tx=0x7f56e4006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:28.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.762+0000 7f56f2ffd640 1 -- 172.21.15.7:0/3358062669 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f56ec0599b0 con 0x7f56f41564e0 2024-09-15T23:36:28.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.973+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56f40b1250 con 0x7f56f41564e0 2024-09-15T23:36:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.973+0000 7f56f2ffd640 1 -- 172.21.15.7:0/3358062669 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5210 (secure 0 0 0) 0x7f56ec0592b0 con 0x7f56f41564e0 2024-09-15T23:36:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:28.974 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":6,"btime":"2024-09-15T23:36:28:618453+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:27.958498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:36:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 6 2024-09-15T23:36:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f56e006c2b0 msgr2=0x7f56e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f56e006c2b0 0x7f56e006e770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f56e4001560 tx=0x7f56e4006040 comp rx=0 tx=0).stop 2024-09-15T23:36:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56f41564e0 msgr2=0x7f56f4156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56f41564e0 0x7f56f4156940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56f40d43b0 tx=0x7f56ec011c30 comp rx=0 tx=0).stop 2024-09-15T23:36:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 shutdown_connections 2024-09-15T23:36:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56f41564e0 0x7f56f4156940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f40ca430 0x7f56f40ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f56e006c2b0 0x7f56e006e770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 --2- 172.21.15.7:0/3358062669 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56dc0043b0 0x7f56f406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 >> 172.21.15.7:0/3358062669 conn(0x7f56f408bfd0 msgr2=0x7f56f406ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 shutdown_connections 2024-09-15T23:36:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:28.976+0000 7f56fc6e4640 1 -- 172.21.15.7:0/3358062669 wait complete. 2024-09-15T23:36:28.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.088+0000 7fecf4f5f640 1 Processor -- start 2024-09-15T23:36:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.088+0000 7fecf4f5f640 1 -- start start 2024-09-15T23:36:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7fecf4f5f640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecf00d0d00 0x7fecf006efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7fecf4f5f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf006dfa0 con 0x7fecf00b31c0 2024-09-15T23:36:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7fecf4f5f640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf0159240 con 0x7fecf0158e60 2024-09-15T23:36:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7fecf4f5f640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fecf006e210 con 0x7fecf00d0d00 2024-09-15T23:36:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7feceed9d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecf00d0d00 0x7fecf006efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7fece7fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fecf00b31c0 0x7fecf00d05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44704/0 (socket says 172.21.15.7:44704) 2024-09-15T23:36:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7fece7fff640 1 -- 172.21.15.7:0/3524510453 learned_addr learned my addr 172.21.15.7:0/3524510453 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7feceed9d640 1 -- 172.21.15.7:0/3524510453 >> v1:172.21.15.18:6789/0 conn(0x7fecf0158e60 legacy=0x7fecf00b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7feceed9d640 1 -- 172.21.15.7:0/3524510453 >> v1:172.21.15.7:6789/0 conn(0x7fecf00b31c0 legacy=0x7fecf00d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7fecedd9b640 1 -- 172.21.15.7:0/3524510453 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070776026 0 0) 0x7fecf006dfa0 con 0x7fecf00b31c0 2024-09-15T23:36:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.089+0000 7feceed9d640 1 -- 172.21.15.7:0/3524510453 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf006e4f0 con 0x7fecf00d0d00 2024-09-15T23:36:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.090+0000 7feceed9d640 1 --2- 172.21.15.7:0/3524510453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf00d0d00 0x7fecf006efd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fece8009d90 tx=0x7fece802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15a7516ce1404e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.090+0000 7fecedd9b640 1 -- 172.21.15.7:0/3524510453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8037070 con 0x7fecf00d0d00 2024-09-15T23:36:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.090+0000 7fecedd9b640 1 -- 172.21.15.7:0/3524510453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece8007740 con 0x7fecf00d0d00 2024-09-15T23:36:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.090+0000 7fecedd9b640 1 -- 172.21.15.7:0/3524510453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8007b10 con 0x7fecf00d0d00 2024-09-15T23:36:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.091+0000 7fecf4f5f640 1 -- 172.21.15.7:0/3524510453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf00d0d00 msgr2=0x7fecf006efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.091+0000 7fecf4f5f640 1 --2- 172.21.15.7:0/3524510453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf00d0d00 0x7fecf006efd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fece8009d90 tx=0x7fece802a780 comp rx=0 tx=0).stop 2024-09-15T23:36:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.091+0000 7fecf4f5f640 1 -- 172.21.15.7:0/3524510453 shutdown_connections 2024-09-15T23:36:29.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.091+0000 7fecf4f5f640 1 --2- 172.21.15.7:0/3524510453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf00d0d00 0x7fecf006efd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.091+0000 7fecf4f5f640 1 -- 172.21.15.7:0/3524510453 >> 172.21.15.7:0/3524510453 conn(0x7fecf00b21e0 msgr2=0x7fecf00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:29.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.091+0000 7fecf4f5f640 1 -- 172.21.15.7:0/3524510453 shutdown_connections 2024-09-15T23:36:29.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.091+0000 7fecf4f5f640 1 -- 172.21.15.7:0/3524510453 wait complete. 2024-09-15T23:36:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.092+0000 7fecf4f5f640 1 Processor -- start 2024-09-15T23:36:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.092+0000 7fecf4f5f640 1 -- start start 2024-09-15T23:36:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7fecf4f5f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fecf00b31c0 0x7fecf0147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7fecf4f5f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fecf00d0d00 0x7fecf01f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7fecf4f5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf0158e60 0x7fecf01552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7fecf4f5f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fecf00b3d90 con 0x7fecf0158e60 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7fecf4f5f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fecf006eab0 con 0x7fecf00d0d00 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7fecf4f5f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fecf00b3f10 con 0x7fecf00b31c0 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7feceed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf0158e60 0x7fecf01552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7feceed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf0158e60 0x7fecf01552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53604/0 (socket says 172.21.15.7:53604) 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7feceed9d640 1 -- 172.21.15.7:0/1171994423 learned_addr learned my addr 172.21.15.7:0/1171994423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7fece7fff640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fecf00d0d00 0x7fecf01f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7fecee59c640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fecf00b31c0 0x7fecf0147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7feceed9d640 1 -- 172.21.15.7:0/1171994423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fecf00b31c0 msgr2=0x7fecf0147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7feceed9d640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fecf00b31c0 0x7fecf0147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7feceed9d640 1 -- 172.21.15.7:0/1171994423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fecf00d0d00 msgr2=0x7fecf01f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7feceed9d640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fecf00d0d00 0x7fecf01f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7feceed9d640 1 -- 172.21.15.7:0/1171994423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf009b3f0 con 0x7fecf0158e60 2024-09-15T23:36:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.093+0000 7feceed9d640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf0158e60 0x7fecf01552a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fece8009d90 tx=0x7fece8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.094+0000 7fece6ffd640 1 -- 172.21.15.7:0/1171994423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8042020 con 0x7fecf0158e60 2024-09-15T23:36:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.094+0000 7fece6ffd640 1 -- 172.21.15.7:0/1171994423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece8032990 con 0x7fecf0158e60 2024-09-15T23:36:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.094+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf0148390 con 0x7fecf0158e60 2024-09-15T23:36:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.094+0000 7fece6ffd640 1 -- 172.21.15.7:0/1171994423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8037040 con 0x7fecf0158e60 2024-09-15T23:36:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.094+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf0145600 con 0x7fecf0158e60 2024-09-15T23:36:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.095+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf015cb40 con 0x7fecf0158e60 2024-09-15T23:36:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.095+0000 7fece6ffd640 1 -- 172.21.15.7:0/1171994423 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fece8045020 con 0x7fecf0158e60 2024-09-15T23:36:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.096+0000 7fece6ffd640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fecd006c2b0 0x7fecd006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.096+0000 7fecee59c640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fecd006c2b0 0x7fecd006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.096+0000 7fece6ffd640 1 -- 172.21.15.7:0/1171994423 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fece80ac580 con 0x7fecf0158e60 2024-09-15T23:36:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.096+0000 7fecee59c640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fecd006c2b0 0x7fecd006e770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fece0000f60 tx=0x7fece0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:29.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.100+0000 7fece6ffd640 1 -- 172.21.15.7:0/1171994423 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fece804c070 con 0x7fecf0158e60 2024-09-15T23:36:29.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.276+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecf00b3740 con 0x7fecf0158e60 2024-09-15T23:36:29.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.277+0000 7fece6ffd640 1 -- 172.21.15.7:0/1171994423 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7fecf0148390 con 0x7fecf0158e60 2024-09-15T23:36:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:29.278 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:29.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fecd006c2b0 msgr2=0x7fecd006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fecd006c2b0 0x7fecd006e770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fece0000f60 tx=0x7fece0009040 comp rx=0 tx=0).stop 2024-09-15T23:36:29.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf0158e60 msgr2=0x7fecf01552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf0158e60 0x7fecf01552a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fece8009d90 tx=0x7fece8004a40 comp rx=0 tx=0).stop 2024-09-15T23:36:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 shutdown_connections 2024-09-15T23:36:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecf0158e60 0x7fecf01552a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fecf00d0d00 0x7fecf01f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fecd006c2b0 0x7fecd006e770 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 --2- 172.21.15.7:0/1171994423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fecf00b31c0 0x7fecf0147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 >> 172.21.15.7:0/1171994423 conn(0x7fecf00b21e0 msgr2=0x7fecf015d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 shutdown_connections 2024-09-15T23:36:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.279+0000 7fecf4f5f640 1 -- 172.21.15.7:0/1171994423 wait complete. 2024-09-15T23:36:29.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.392+0000 7fe7bdfbd640 1 Processor -- start 2024-09-15T23:36:29.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.393+0000 7fe7bdfbd640 1 -- start start 2024-09-15T23:36:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.393+0000 7fe7bdfbd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe7b8161e90 0x7fe7b809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.393+0000 7fe7bdfbd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7b80b3990 0x7fe7b809af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.393+0000 7fe7bdfbd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b80d3100 con 0x7fe7b809b4c0 2024-09-15T23:36:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.393+0000 7fe7bdfbd640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b80d1240 con 0x7fe7b80b3990 2024-09-15T23:36:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.393+0000 7fe7bdfbd640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b80d0ed0 con 0x7fe7b8161e90 2024-09-15T23:36:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.394+0000 7fe7b6ffd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7b80b3990 0x7fe7b809af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.394+0000 7fe7b6ffd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7b80b3990 0x7fe7b809af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53620/0 (socket says 172.21.15.7:53620) 2024-09-15T23:36:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.394+0000 7fe7b6ffd640 1 -- 172.21.15.7:0/2282445077 learned_addr learned my addr 172.21.15.7:0/2282445077 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.394+0000 7fe7b77fe640 1 --2- 172.21.15.7:0/2282445077 >> v2:172.21.15.18:3300/0 conn(0x7fe7b8161e90 0x7fe7b809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.394+0000 7fe7b6ffd640 1 -- 172.21.15.7:0/2282445077 >> v2:172.21.15.18:3300/0 conn(0x7fe7b8161e90 msgr2=0x7fe7b809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.394+0000 7fe7b6ffd640 1 --2- 172.21.15.7:0/2282445077 >> v2:172.21.15.18:3300/0 conn(0x7fe7b8161e90 0x7fe7b809aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.394+0000 7fe7b6ffd640 1 -- 172.21.15.7:0/2282445077 >> v1:172.21.15.7:6789/0 conn(0x7fe7b809b4c0 legacy=0x7fe7b8162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.394+0000 7fe7b67fc640 1 -- 172.21.15.7:0/2282445077 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3918510133 0 0) 0x7fe7b80d3100 con 0x7fe7b809b4c0 2024-09-15T23:36:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.394+0000 7fe7b6ffd640 1 -- 172.21.15.7:0/2282445077 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b8162270 con 0x7fe7b80b3990 2024-09-15T23:36:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7b6ffd640 1 --2- 172.21.15.7:0/2282445077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b80b3990 0x7fe7b809af80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80733c0 tx=0x7fe7ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd6fe34503e2d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7b67fc640 1 -- 172.21.15.7:0/2282445077 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac037070 con 0x7fe7b80b3990 2024-09-15T23:36:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7b67fc640 1 -- 172.21.15.7:0/2282445077 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ac032440 con 0x7fe7b80b3990 2024-09-15T23:36:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7b67fc640 1 -- 172.21.15.7:0/2282445077 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac032810 con 0x7fe7b80b3990 2024-09-15T23:36:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/2282445077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b80b3990 msgr2=0x7fe7b809af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7bdfbd640 1 --2- 172.21.15.7:0/2282445077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b80b3990 0x7fe7b809af80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80733c0 tx=0x7fe7ac02a1e0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/2282445077 shutdown_connections 2024-09-15T23:36:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7bdfbd640 1 --2- 172.21.15.7:0/2282445077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b80b3990 0x7fe7b809af80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7bdfbd640 1 --2- 172.21.15.7:0/2282445077 >> v2:172.21.15.18:3300/0 conn(0x7fe7b8161e90 0x7fe7b809aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/2282445077 >> 172.21.15.7:0/2282445077 conn(0x7fe7b80d2440 msgr2=0x7fe7b80d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/2282445077 shutdown_connections 2024-09-15T23:36:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.395+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/2282445077 wait complete. 2024-09-15T23:36:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.396+0000 7fe7bdfbd640 1 Processor -- start 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.397+0000 7fe7bdfbd640 1 -- start start 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.397+0000 7fe7bdfbd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b809b4c0 0x7fe7b80cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.397+0000 7fe7bdfbd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7b80b3990 0x7fe7b814d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.397+0000 7fe7bdfbd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b8161e90 0x7fe7b80b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.397+0000 7fe7bdfbd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b806e520 con 0x7fe7b809b4c0 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.397+0000 7fe7bdfbd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b806e3a0 con 0x7fe7b8161e90 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.397+0000 7fe7bdfbd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b806e6a0 con 0x7fe7b80b3990 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b809b4c0 0x7fe7b80cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b809b4c0 0x7fe7b80cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53624/0 (socket says 172.21.15.7:53624) 2024-09-15T23:36:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b77fe640 1 -- 172.21.15.7:0/3290452745 learned_addr learned my addr 172.21.15.7:0/3290452745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b77fe640 1 -- 172.21.15.7:0/3290452745 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7b80b3990 msgr2=0x7fe7b814d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b6ffd640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7b80b3990 0x7fe7b814d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b7fff640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b8161e90 0x7fe7b80b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b77fe640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7b80b3990 0x7fe7b814d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b77fe640 1 -- 172.21.15.7:0/3290452745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b8161e90 msgr2=0x7fe7b80b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b77fe640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b8161e90 0x7fe7b80b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b77fe640 1 -- 172.21.15.7:0/3290452745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b814d6f0 con 0x7fe7b809b4c0 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7b77fe640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b809b4c0 0x7fe7b80cbb00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800e8a0 tx=0x7fe7a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe79ffff640 1 -- 172.21.15.7:0/3290452745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8017070 con 0x7fe7b809b4c0 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe79ffff640 1 -- 172.21.15.7:0/3290452745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a800c940 con 0x7fe7b809b4c0 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b80caa60 con 0x7fe7b809b4c0 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe79ffff640 1 -- 172.21.15.7:0/3290452745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800cd30 con 0x7fe7b809b4c0 2024-09-15T23:36:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.398+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b80cb020 con 0x7fe7b809b4c0 2024-09-15T23:36:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.399+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b80d3100 con 0x7fe7b809b4c0 2024-09-15T23:36:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.400+0000 7fe79ffff640 1 -- 172.21.15.7:0/3290452745 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe7a801a070 con 0x7fe7b809b4c0 2024-09-15T23:36:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.400+0000 7fe79ffff640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe7a406c2b0 0x7fe7a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.400+0000 7fe79ffff640 1 -- 172.21.15.7:0/3290452745 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe7a808b590 con 0x7fe7b809b4c0 2024-09-15T23:36:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.400+0000 7fe7b6ffd640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe7a406c2b0 0x7fe7a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.401+0000 7fe7b6ffd640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe7a406c2b0 0x7fe7a406e770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac034000 tx=0x7fe7ac02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:29.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.405+0000 7fe79ffff640 1 -- 172.21.15.7:0/3290452745 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe7b814d6f0 con 0x7fe7b809b4c0 2024-09-15T23:36:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.579+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7b815df00 con 0x7fe7b809b4c0 2024-09-15T23:36:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.580+0000 7fe79ffff640 1 -- 172.21.15.7:0/3290452745 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7fe7a8059a50 con 0x7fe7b809b4c0 2024-09-15T23:36:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe7a406c2b0 msgr2=0x7fe7a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe7a406c2b0 0x7fe7a406e770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac034000 tx=0x7fe7ac02e000 comp rx=0 tx=0).stop 2024-09-15T23:36:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b809b4c0 msgr2=0x7fe7b80cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b809b4c0 0x7fe7b80cbb00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800e8a0 tx=0x7fe7a800ed60 comp rx=0 tx=0).stop 2024-09-15T23:36:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 shutdown_connections 2024-09-15T23:36:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b8161e90 0x7fe7b80b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe7a406c2b0 0x7fe7a406e770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7b80b3990 0x7fe7b814d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 --2- 172.21.15.7:0/3290452745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b809b4c0 0x7fe7b80cbb00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 >> 172.21.15.7:0/3290452745 conn(0x7fe7b80d2440 msgr2=0x7fe7b80cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 shutdown_connections 2024-09-15T23:36:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.583+0000 7fe7bdfbd640 1 -- 172.21.15.7:0/3290452745 wait complete. 2024-09-15T23:36:29.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:36:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.712+0000 7f5190847640 1 Processor -- start 2024-09-15T23:36:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.712+0000 7f5190847640 1 -- start start 2024-09-15T23:36:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f5190847640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f518c15e6d0 0x7f518c0d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f5190847640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f518c0d32a0 0x7f518c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f5190847640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f518c0b3d10 con 0x7f518c0d51c0 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f5190847640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f518c0cfcd0 con 0x7f518c15e6d0 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f5190847640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f518c0cff50 con 0x7f518c0d32a0 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f518a59c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f518c15e6d0 0x7f518c0d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f5189d9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f518c0d51c0 0x7f518c0d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44714/0 (socket says 172.21.15.7:44714) 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f518a59c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f518c15e6d0 0x7f518c0d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53632/0 (socket says 172.21.15.7:53632) 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f5189d9b640 1 -- 172.21.15.7:0/651943610 learned_addr learned my addr 172.21.15.7:0/651943610 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f518ad9d640 1 --2- 172.21.15.7:0/651943610 >> v2:172.21.15.18:3300/0 conn(0x7f518c0d32a0 0x7f518c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f518a59c640 1 -- 172.21.15.7:0/651943610 >> v2:172.21.15.18:3300/0 conn(0x7f518c0d32a0 msgr2=0x7f518c0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f518a59c640 1 --2- 172.21.15.7:0/651943610 >> v2:172.21.15.18:3300/0 conn(0x7f518c0d32a0 0x7f518c0b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f518a59c640 1 -- 172.21.15.7:0/651943610 >> v1:172.21.15.7:6789/0 conn(0x7f518c0d51c0 legacy=0x7f518c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f518a59c640 1 -- 172.21.15.7:0/651943610 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c0d0230 con 0x7f518c15e6d0 2024-09-15T23:36:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.713+0000 7f518959a640 1 -- 172.21.15.7:0/651943610 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3243685216 0 0) 0x7f518c0b3d10 con 0x7f518c0d51c0 2024-09-15T23:36:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.714+0000 7f518a59c640 1 --2- 172.21.15.7:0/651943610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c15e6d0 0x7f518c0d21b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f518c0d3800 tx=0x7f518402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8aff49f6d1fd7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.714+0000 7f518959a640 1 -- 172.21.15.7:0/651943610 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184037070 con 0x7f518c15e6d0 2024-09-15T23:36:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.714+0000 7f518959a640 1 -- 172.21.15.7:0/651943610 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5184032440 con 0x7f518c15e6d0 2024-09-15T23:36:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.714+0000 7f518959a640 1 -- 172.21.15.7:0/651943610 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184032810 con 0x7f518c15e6d0 2024-09-15T23:36:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.715+0000 7f5190847640 1 -- 172.21.15.7:0/651943610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c15e6d0 msgr2=0x7f518c0d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.715+0000 7f5190847640 1 --2- 172.21.15.7:0/651943610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c15e6d0 0x7f518c0d21b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f518c0d3800 tx=0x7f518402a1a0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.715+0000 7f5190847640 1 -- 172.21.15.7:0/651943610 shutdown_connections 2024-09-15T23:36:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.715+0000 7f5190847640 1 --2- 172.21.15.7:0/651943610 >> v2:172.21.15.18:3300/0 conn(0x7f518c0d32a0 0x7f518c0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.715+0000 7f5190847640 1 --2- 172.21.15.7:0/651943610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c15e6d0 0x7f518c0d21b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.715+0000 7f5190847640 1 -- 172.21.15.7:0/651943610 >> 172.21.15.7:0/651943610 conn(0x7f518c08bfd0 msgr2=0x7f518c0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:29.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.715+0000 7f5190847640 1 -- 172.21.15.7:0/651943610 shutdown_connections 2024-09-15T23:36:29.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.715+0000 7f5190847640 1 -- 172.21.15.7:0/651943610 wait complete. 2024-09-15T23:36:29.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.716+0000 7f5190847640 1 Processor -- start 2024-09-15T23:36:29.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.716+0000 7f5190847640 1 -- start start 2024-09-15T23:36:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f5190847640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c0d32a0 0x7f518c15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f5190847640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f518c0d51c0 0x7f518c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f5190847640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f518c15e6d0 0x7f518c0b9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f5190847640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f518c0d26f0 con 0x7f518c0d32a0 2024-09-15T23:36:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f5190847640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f518c0d0510 con 0x7f518c15e6d0 2024-09-15T23:36:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f5190847640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f518c0d2870 con 0x7f518c0d51c0 2024-09-15T23:36:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f518a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c0d32a0 0x7f518c15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f518a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c0d32a0 0x7f518c15dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53648/0 (socket says 172.21.15.7:53648) 2024-09-15T23:36:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f518a59c640 1 -- 172.21.15.7:0/1426302597 learned_addr learned my addr 172.21.15.7:0/1426302597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f5189d9b640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f518c0d51c0 0x7f518c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f518ad9d640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f518c15e6d0 0x7f518c0b9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f518a59c640 1 -- 172.21.15.7:0/1426302597 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f518c0d51c0 msgr2=0x7f518c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f518a59c640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f518c0d51c0 0x7f518c15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f518a59c640 1 -- 172.21.15.7:0/1426302597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f518c15e6d0 msgr2=0x7f518c0b9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f518a59c640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f518c15e6d0 0x7f518c0b9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.717+0000 7f518a59c640 1 -- 172.21.15.7:0/1426302597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c15f7b0 con 0x7f518c0d32a0 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.718+0000 7f518a59c640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c0d32a0 0x7f518c15dbf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5184035040 tx=0x7f51840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.718+0000 7f5182ffd640 1 -- 172.21.15.7:0/1426302597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184037050 con 0x7f518c0d32a0 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.718+0000 7f5182ffd640 1 -- 172.21.15.7:0/1426302597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5184038040 con 0x7f518c0d32a0 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.718+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c15fa90 con 0x7f518c0d32a0 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.718+0000 7f5182ffd640 1 -- 172.21.15.7:0/1426302597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184006e10 con 0x7f518c0d32a0 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.718+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c0b9750 con 0x7f518c0d32a0 2024-09-15T23:36:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.719+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518c0b41c0 con 0x7f518c0d32a0 2024-09-15T23:36:29.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.719+0000 7f5182ffd640 1 -- 172.21.15.7:0/1426302597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5184030980 con 0x7f518c0d32a0 2024-09-15T23:36:29.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.720+0000 7f5182ffd640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f516c06c2b0 0x7f516c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:29.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.720+0000 7f5182ffd640 1 -- 172.21.15.7:0/1426302597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f51840ac1f0 con 0x7f518c0d32a0 2024-09-15T23:36:29.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.720+0000 7f5189d9b640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f516c06c2b0 0x7f516c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:29.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.720+0000 7f5189d9b640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f516c06c2b0 0x7f516c06e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f518c161860 tx=0x7f5174007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:29.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.724+0000 7f5182ffd640 1 -- 172.21.15.7:0/1426302597 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f518407a6b0 con 0x7f518c0d32a0 2024-09-15T23:36:29.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.934+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f518c15e130 con 0x7f518c0d32a0 2024-09-15T23:36:29.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.934+0000 7f5182ffd640 1 -- 172.21.15.7:0/1426302597 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5210 (secure 0 0 0) 0x7f5184079fb0 con 0x7f518c0d32a0 2024-09-15T23:36:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:29.935 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":6,"btime":"2024-09-15T23:36:28:618453+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:27.958498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:36:29.936 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 6 2024-09-15T23:36:29.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.936+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f516c06c2b0 msgr2=0x7f516c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.936+0000 7f5190847640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f516c06c2b0 0x7f516c06e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f518c161860 tx=0x7f5174007b50 comp rx=0 tx=0).stop 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.936+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c0d32a0 msgr2=0x7f518c15dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.936+0000 7f5190847640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c0d32a0 0x7f518c15dbf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5184035040 tx=0x7f51840046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.937+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 shutdown_connections 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.937+0000 7f5190847640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f518c15e6d0 0x7f518c0b9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.937+0000 7f5190847640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f516c06c2b0 0x7f516c06e770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.937+0000 7f5190847640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f518c0d51c0 0x7f518c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.937+0000 7f5190847640 1 --2- 172.21.15.7:0/1426302597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518c0d32a0 0x7f518c15dbf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.937+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 >> 172.21.15.7:0/1426302597 conn(0x7f518c08bfd0 msgr2=0x7f518c1e8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.937+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 shutdown_connections 2024-09-15T23:36:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:29.937+0000 7f5190847640 1 -- 172.21.15.7:0/1426302597 wait complete. 2024-09-15T23:36:29.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.051+0000 7f77e061e640 1 Processor -- start 2024-09-15T23:36:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.051+0000 7f77e061e640 1 -- start start 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77e061e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d815e6d0 0x7f77d80d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77e061e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f77d80d51c0 0x7f77d80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77e061e640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d80b3d10 con 0x7f77d80d30d0 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77e061e640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f77d80d3540 con 0x7f77d80d51c0 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77e061e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f77d80cfcd0 con 0x7f77d815e6d0 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77ddbb9640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f77d80d51c0 0x7f77d80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77debbb640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f77d80d30d0 0x7f77d80d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44728/0 (socket says 172.21.15.7:44728) 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77ddbb9640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f77d80d51c0 0x7f77d80d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53664/0 (socket says 172.21.15.7:53664) 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77debbb640 1 -- 172.21.15.7:0/1490127955 learned_addr learned my addr 172.21.15.7:0/1490127955 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77de3ba640 1 --2- 172.21.15.7:0/1490127955 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d815e6d0 0x7f77d80d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77ddbb9640 1 -- 172.21.15.7:0/1490127955 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d815e6d0 msgr2=0x7f77d80d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77ddbb9640 1 --2- 172.21.15.7:0/1490127955 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d815e6d0 0x7f77d80d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77ddbb9640 1 -- 172.21.15.7:0/1490127955 >> v1:172.21.15.7:6789/0 conn(0x7f77d80d30d0 legacy=0x7f77d80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.052+0000 7f77ddbb9640 1 -- 172.21.15.7:0/1490127955 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d80cff50 con 0x7f77d80d51c0 2024-09-15T23:36:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.053+0000 7f77ddbb9640 1 --2- 172.21.15.7:0/1490127955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d80d51c0 0x7f77d80d2b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f77c80097a0 tx=0x7f77c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f69d5407cb86a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.053+0000 7f77dd3b8640 1 -- 172.21.15.7:0/1490127955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8037070 con 0x7f77d80d51c0 2024-09-15T23:36:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.053+0000 7f77dd3b8640 1 -- 172.21.15.7:0/1490127955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c8032440 con 0x7f77d80d51c0 2024-09-15T23:36:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.053+0000 7f77dd3b8640 1 -- 172.21.15.7:0/1490127955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8032810 con 0x7f77d80d51c0 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.054+0000 7f77e061e640 1 -- 172.21.15.7:0/1490127955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d80d51c0 msgr2=0x7f77d80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.054+0000 7f77e061e640 1 --2- 172.21.15.7:0/1490127955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d80d51c0 0x7f77d80d2b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f77c80097a0 tx=0x7f77c802a180 comp rx=0 tx=0).stop 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.054+0000 7f77e061e640 1 -- 172.21.15.7:0/1490127955 shutdown_connections 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.054+0000 7f77e061e640 1 --2- 172.21.15.7:0/1490127955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d80d51c0 0x7f77d80d2b90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.054+0000 7f77e061e640 1 --2- 172.21.15.7:0/1490127955 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d815e6d0 0x7f77d80d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.054+0000 7f77e061e640 1 -- 172.21.15.7:0/1490127955 >> 172.21.15.7:0/1490127955 conn(0x7f77d808bfd0 msgr2=0x7f77d80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.054+0000 7f77e061e640 1 -- 172.21.15.7:0/1490127955 shutdown_connections 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.054+0000 7f77e061e640 1 -- 172.21.15.7:0/1490127955 wait complete. 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.055+0000 7f77e061e640 1 Processor -- start 2024-09-15T23:36:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.055+0000 7f77e061e640 1 -- start start 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77e061e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d80d30d0 0x7f77d80b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77e061e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d80d51c0 0x7f77d8155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77e061e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d815e6d0 0x7f77d815fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77e061e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80d0510 con 0x7f77d815e6d0 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77e061e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f77d815eab0 con 0x7f77d80d51c0 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77e061e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f77d80d26f0 con 0x7f77d80d30d0 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77debbb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d815e6d0 0x7f77d815fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77debbb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d815e6d0 0x7f77d815fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53672/0 (socket says 172.21.15.7:53672) 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77debbb640 1 -- 172.21.15.7:0/3708128275 learned_addr learned my addr 172.21.15.7:0/3708128275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77de3ba640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d80d30d0 0x7f77d80b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77ddbb9640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d80d51c0 0x7f77d8155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77debbb640 1 -- 172.21.15.7:0/3708128275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d80d30d0 msgr2=0x7f77d80b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77debbb640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d80d30d0 0x7f77d80b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77debbb640 1 -- 172.21.15.7:0/3708128275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d80d51c0 msgr2=0x7f77d8155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77debbb640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d80d51c0 0x7f77d8155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77debbb640 1 -- 172.21.15.7:0/3708128275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d8155ec0 con 0x7f77d815e6d0 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77ddbb9640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d80d51c0 0x7f77d8155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77debbb640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d815e6d0 0x7f77d815fe50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f77d000cbc0 tx=0x7f77d000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77ceffd640 1 -- 172.21.15.7:0/3708128275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0017070 con 0x7f77d815e6d0 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77ceffd640 1 -- 172.21.15.7:0/3708128275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d000ad40 con 0x7f77d815e6d0 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d80b79f0 con 0x7f77d815e6d0 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77ceffd640 1 -- 172.21.15.7:0/3708128275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0012600 con 0x7f77d815e6d0 2024-09-15T23:36:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.056+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d80b7fb0 con 0x7f77d815e6d0 2024-09-15T23:36:30.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.057+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d80d2b90 con 0x7f77d815e6d0 2024-09-15T23:36:30.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.058+0000 7f77ceffd640 1 -- 172.21.15.7:0/3708128275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f77d001a070 con 0x7f77d815e6d0 2024-09-15T23:36:30.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.058+0000 7f77ceffd640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f77bc06c2b0 0x7f77bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.059+0000 7f77de3ba640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f77bc06c2b0 0x7f77bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.059+0000 7f77ceffd640 1 -- 172.21.15.7:0/3708128275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f77d008b4a0 con 0x7f77d815e6d0 2024-09-15T23:36:30.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.059+0000 7f77de3ba640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f77bc06c2b0 0x7f77bc06e770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009aa0 tx=0x7f77d4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.063+0000 7f77ceffd640 1 -- 172.21.15.7:0/3708128275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f77d00591e0 con 0x7f77d815e6d0 2024-09-15T23:36:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.237+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77d80b8720 con 0x7f77d815e6d0 2024-09-15T23:36:30.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.238+0000 7f77ceffd640 1 -- 172.21.15.7:0/3708128275 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7f77d005d4e0 con 0x7f77d815e6d0 2024-09-15T23:36:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:30.239 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:30.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f77bc06c2b0 msgr2=0x7f77bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f77bc06c2b0 0x7f77bc06e770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009aa0 tx=0x7f77d4009190 comp rx=0 tx=0).stop 2024-09-15T23:36:30.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d815e6d0 msgr2=0x7f77d815fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d815e6d0 0x7f77d815fe50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f77d000cbc0 tx=0x7f77d000a430 comp rx=0 tx=0).stop 2024-09-15T23:36:30.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 shutdown_connections 2024-09-15T23:36:30.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77d815e6d0 0x7f77d815fe50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d80d51c0 0x7f77d8155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f77bc06c2b0 0x7f77bc06e770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 --2- 172.21.15.7:0/3708128275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f77d80d30d0 0x7f77d80b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.240+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 >> 172.21.15.7:0/3708128275 conn(0x7f77d808bfd0 msgr2=0x7f77d80b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:30.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.241+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 shutdown_connections 2024-09-15T23:36:30.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.241+0000 7f77e061e640 1 -- 172.21.15.7:0/3708128275 wait complete. 2024-09-15T23:36:30.654 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-15T23:36:22.916234+0000', 'modified': '2024-09-15T23:36:27.958498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.18:6833/3035612480', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6832', 'nonce': 3035612480}, {'type': 'v1', 'addr': '172.21.15.18:6833', 'nonce': 3035612480}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4291, 'qdb_cluster': [4291]} 2024-09-15T23:36:30.655 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-15T23:36:30.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-15T23:36:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.761+0000 7f46098dd640 1 Processor -- start 2024-09-15T23:36:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.761+0000 7f46098dd640 1 -- start start 2024-09-15T23:36:30.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f46098dd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f460415e5b0 0x7f4604162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f46098dd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46040d2980 con 0x7f46040d24e0 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f46098dd640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46040d2c40 con 0x7f460415e990 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f46098dd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f46040d2f20 con 0x7f460415e5b0 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f46027fc640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f46040d24e0 0x7f4604162890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44742/0 (socket says 172.21.15.7:44742) 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f46027fc640 1 -- 172.21.15.7:0/2496323779 learned_addr learned my addr 172.21.15.7:0/2496323779 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4602ffd640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f460415e5b0 0x7f4604162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 640080497 0 0) 0x7f46040d2980 con 0x7f46040d24e0 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45ec003520 con 0x7f46040d24e0 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2755379452 0 0) 0x7f46040d2c40 con 0x7f460415e990 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46040d2980 con 0x7f460415e990 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3587779969 0 0) 0x7f45ec003520 con 0x7f46040d24e0 2024-09-15T23:36:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f460415e5b0 msgr2=0x7f4604162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f460415e5b0 0x7f4604162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 >> v1:172.21.15.18:6789/0 conn(0x7f460415e990 legacy=0x7f46040cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46040d3200 con 0x7f46040d24e0 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f45f80017f0 con 0x7f46040d24e0 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 >> v1:172.21.15.7:6789/0 conn(0x7f46040d24e0 legacy=0x7f4604162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f45ec003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45ec004800 0x7f45ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.762+0000 7f4601ffb640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f460415e5b0 0x7f45ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46040d2f20 con 0x7f45ec003b70 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00ba70 con 0x7f45ec004800 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f45ec00bc40 con 0x7f460415e5b0 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f4602ffd640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f45ec003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f46027fc640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45ec004800 0x7f45ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f46037fe640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f460415e5b0 0x7f45ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f4602ffd640 1 -- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f460415e5b0 msgr2=0x7f45ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f4602ffd640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f460415e5b0 0x7f45ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f4602ffd640 1 -- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45ec004800 msgr2=0x7f45ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f4602ffd640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45ec004800 0x7f45ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f4602ffd640 1 -- 172.21.15.7:0/2496323779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec003520 con 0x7f45ec003b70 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f46027fc640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45ec004800 0x7f45ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.763+0000 7f46037fe640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f460415e5b0 0x7f45ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f4602ffd640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f45ec003f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009ac0 tx=0x7f45ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918924dbbf839c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4036070 con 0x7f45ec003b70 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f4031400 con 0x7f45ec003b70 2024-09-15T23:36:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f4601ffb640 1 -- 172.21.15.7:0/2496323779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4009690 con 0x7f45ec003b70 2024-09-15T23:36:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f46098dd640 1 -- 172.21.15.7:0/2496323779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 msgr2=0x7f45ec003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f46098dd640 1 --2- 172.21.15.7:0/2496323779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f45ec003f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009ac0 tx=0x7f45ec005830 comp rx=0 tx=0).stop 2024-09-15T23:36:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f46037fe640 1 -- 172.21.15.7:0/2496323779 reap_dead start 2024-09-15T23:36:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f46098dd640 1 -- 172.21.15.7:0/2496323779 shutdown_connections 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f46098dd640 1 -- 172.21.15.7:0/2496323779 >> 172.21.15.7:0/2496323779 conn(0x7f460408bfd0 msgr2=0x7f46040d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f46098dd640 1 -- 172.21.15.7:0/2496323779 shutdown_connections 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.764+0000 7f46098dd640 1 -- 172.21.15.7:0/2496323779 wait complete. 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.765+0000 7f46098dd640 1 Processor -- start 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.766+0000 7f46098dd640 1 -- start start 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.766+0000 7f46098dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f46040b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.766+0000 7f46098dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4604153160 0x7f46041f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.766+0000 7f46098dd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f46040b5090 0x7f46041f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.766+0000 7f46098dd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46040b3b40 con 0x7f45ec003b70 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.766+0000 7f46098dd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f46040b39c0 con 0x7f4604153160 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.766+0000 7f46098dd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f4604162fa0 con 0x7f46040b5090 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f4602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f46040b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f4602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f46040b9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53696/0 (socket says 172.21.15.7:53696) 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f4602ffd640 1 -- 172.21.15.7:0/1767824174 learned_addr learned my addr 172.21.15.7:0/1767824174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f4602ffd640 1 -- 172.21.15.7:0/1767824174 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f46040b5090 msgr2=0x7f46041f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f4602ffd640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f46040b5090 0x7f46041f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f4602ffd640 1 -- 172.21.15.7:0/1767824174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4604153160 msgr2=0x7f46041f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f46027fc640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4604153160 0x7f46041f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f4602ffd640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4604153160 0x7f46041f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f4602ffd640 1 -- 172.21.15.7:0/1767824174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46040c91f0 con 0x7f45ec003b70 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f46027fc640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4604153160 0x7f46041f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f4602ffd640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f46040b9dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034040 tx=0x7f45f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f45ebfff640 1 -- 172.21.15.7:0/1767824174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4041020 con 0x7f45ec003b70 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f45ebfff640 1 -- 172.21.15.7:0/1767824174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f403c470 con 0x7f45ec003b70 2024-09-15T23:36:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f46098dd640 1 -- 172.21.15.7:0/1767824174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46040c94d0 con 0x7f45ec003b70 2024-09-15T23:36:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f45ebfff640 1 -- 172.21.15.7:0/1767824174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4036040 con 0x7f45ec003b70 2024-09-15T23:36:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.767+0000 7f46098dd640 1 -- 172.21.15.7:0/1767824174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46041f51a0 con 0x7f45ec003b70 2024-09-15T23:36:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.769+0000 7f45ebfff640 1 -- 172.21.15.7:0/1767824174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f45f4044020 con 0x7f45ec003b70 2024-09-15T23:36:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.769+0000 7f45ebfff640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f45f006c2e0 0x7f45f006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.769+0000 7f45ebfff640 1 -- 172.21.15.7:0/1767824174 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f45f40ab680 con 0x7f45ec003b70 2024-09-15T23:36:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.769+0000 7f46027fc640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f45f006c2e0 0x7f45f006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.770+0000 7f46027fc640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f45f006c2e0 0x7f45f006e7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45f8001eb0 tx=0x7f45f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.770+0000 7f460c674740 1 Processor -- start 2024-09-15T23:36:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.770+0000 7f460c674740 1 -- start start 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f460c674740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f460c674740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd269a80 0x55f1cd26c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f460c674740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26da50 0x55f1cd2668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f460c674740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f1cd1ce520 con 0x55f1cd26c2c0 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f460c674740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f1cd1763e0 con 0x55f1cd269a80 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f460c674740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x55f1cd1ca4e0 con 0x55f1cd26da50 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f4602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f4602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53702/0 (socket says 172.21.15.7:53702) 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f4602ffd640 1 -- 172.21.15.7:0/3864114322 learned_addr learned my addr 172.21.15.7:0/3864114322 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f46027fc640 1 --2- 172.21.15.7:0/3864114322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd269a80 0x55f1cd26c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f46037fe640 1 --2- 172.21.15.7:0/3864114322 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26da50 0x55f1cd2668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f4602ffd640 1 -- 172.21.15.7:0/3864114322 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26da50 msgr2=0x55f1cd2668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f4602ffd640 1 --2- 172.21.15.7:0/3864114322 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26da50 0x55f1cd2668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f4602ffd640 1 -- 172.21.15.7:0/3864114322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd269a80 msgr2=0x55f1cd26c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f4602ffd640 1 --2- 172.21.15.7:0/3864114322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd269a80 0x55f1cd26c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.771+0000 7f4602ffd640 1 -- 172.21.15.7:0/3864114322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1cd1912f0 con 0x55f1cd26c2c0 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.772+0000 7f4602ffd640 1 --2- 172.21.15.7:0/3864114322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b1a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45f40a97b0 tx=0x7f45f4009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcacd3d67a5fd4d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.772+0000 7f45e97fa640 1 -- 172.21.15.7:0/3864114322 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4081f90 con 0x55f1cd26c2c0 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.772+0000 7f45e97fa640 1 -- 172.21.15.7:0/3864114322 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f4077450 con 0x55f1cd26c2c0 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.772+0000 7f45e97fa640 1 -- 172.21.15.7:0/3864114322 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4075a20 con 0x55f1cd26c2c0 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.772+0000 7f460c674740 1 -- 172.21.15.7:0/3864114322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 msgr2=0x55f1cd2b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.772+0000 7f460c674740 1 --2- 172.21.15.7:0/3864114322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b1a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45f40a97b0 tx=0x7f45f4009890 comp rx=0 tx=0).stop 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.773+0000 7f460c674740 1 -- 172.21.15.7:0/3864114322 shutdown_connections 2024-09-15T23:36:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.773+0000 7f460c674740 1 --2- 172.21.15.7:0/3864114322 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26da50 0x55f1cd2668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.773+0000 7f460c674740 1 --2- 172.21.15.7:0/3864114322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd269a80 0x55f1cd26c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.773+0000 7f460c674740 1 --2- 172.21.15.7:0/3864114322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b1a60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.773+0000 7f460c674740 1 -- 172.21.15.7:0/3864114322 >> 172.21.15.7:0/3864114322 conn(0x55f1cd268a80 msgr2=0x55f1cd2b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.773+0000 7f460c674740 1 -- 172.21.15.7:0/3864114322 shutdown_connections 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.773+0000 7f460c674740 1 -- 172.21.15.7:0/3864114322 wait complete. 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.773+0000 7f460c674740 1 -- start start 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.773+0000 7f460c674740 1 Processor -- start 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f460c674740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 0x55f1cd295740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f460c674740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26da50 0x55f1cd28d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f460c674740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd296890 0x55f1cd2946f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f460c674740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f1cd1ca4e0 con 0x55f1cd26da50 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f460c674740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f1cd157450 con 0x55f1cd296890 2024-09-15T23:36:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f460c674740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x55f1cd1ce520 con 0x55f1cd26c2c0 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f460c674740 10 client.0 _ll_register_callbacks cb 0x55f1ccee0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f46027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26da50 0x55f1cd28d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f46027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26da50 0x55f1cd28d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53712/0 (socket says 172.21.15.7:53712) 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f46027fc640 1 -- 172.21.15.7:0/1971381366 learned_addr learned my addr 172.21.15.7:0/1971381366 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f4602ffd640 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 0x55f1cd295740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f46037fe640 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd296890 0x55f1cd2946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f46027fc640 1 -- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 msgr2=0x55f1cd295740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f46027fc640 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 0x55f1cd295740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f46027fc640 1 -- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd296890 msgr2=0x55f1cd2946f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f46027fc640 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd296890 0x55f1cd2946f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.774+0000 7f46027fc640 1 -- 172.21.15.7:0/1971381366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ccee12d0 con 0x55f1cd26da50 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f46027fc640 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26da50 0x55f1cd28d730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45f800ffe0 tx=0x7f46040d2e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc74eb6a4e7e83d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f45e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f45e27fc640 1 -- 172.21.15.7:0/1971381366 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8010040 con 0x55f1cd26da50 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f45e27fc640 1 -- 172.21.15.7:0/1971381366 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f45f8021630 con 0x55f1cd26da50 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f460c674740 1 -- 172.21.15.7:0/1971381366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f4077450 con 0x55f1cd26da50 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f45e27fc640 1 -- 172.21.15.7:0/1971381366 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f45f802e050 con 0x55f1cd26da50 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f460c674740 10 client.4407 fetch_fsmap learned FSMap version 6 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f460c674740 1 -- 172.21.15.7:0/1971381366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55f1ccfab6b0 con 0x55f1cd26da50 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f45e27fc640 1 -- 172.21.15.7:0/1971381366 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f4077450 con 0x55f1cd26da50 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f45e27fc640 1 -- 172.21.15.7:0/1971381366 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7f45f8031020 con 0x55f1cd26da50 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f460c674740 10 client.4407 fetch_fsmap finished waiting for FSMap version 6 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f460c674740 10 client.4407 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f460c674740 20 client.4407 populate_metadata read hostname 'smithi007' 2024-09-15T23:36:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f460c674740 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55f1cd27dc40 0x55f1cd27e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f460c674740 4 client.4407 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.775+0000 7f460c674740 1 -- 172.21.15.7:0/1971381366 --> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1cd1dbc50 con 0x55f1cd27dc40 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.776+0000 7f4602ffd640 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55f1cd27dc40 0x55f1cd27e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.776+0000 7f45e27fc640 10 client.4407 ms_handle_connect on v2:172.21.15.18:6832/3035612480 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.776+0000 7f4602ffd640 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55f1cd27dc40 0x55f1cd27e020 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91ea322f0562efcd server_cookie=e842d852d8087fb9 in_seq=0 out_seq=0 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f45e27fc640 1 -- 172.21.15.7:0/1971381366 <== mds.0 v2:172.21.15.18:6832/3035612480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1cd1dbc50 con 0x55f1cd27dc40 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f45e27fc640 10 client.4407 handle_command_reply: tid=0 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 client.4407 shutdown 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 -- 172.21.15.7:0/1971381366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26da50 msgr2=0x55f1cd28d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26da50 0x55f1cd28d730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45f800ffe0 tx=0x7f46040d2e10 comp rx=0 tx=0).stop 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 -- 172.21.15.7:0/1971381366 shutdown_connections 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55f1cd27dc40 0x55f1cd27e020 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd296890 0x55f1cd2946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f45e27fc640 0 client.4407 ms_handle_reset on v2:172.21.15.18:6832/3035612480 2024-09-15T23:36:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26da50 0x55f1cd28d730 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 --2- 172.21.15.7:0/1971381366 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 0x55f1cd295740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 -- 172.21.15.7:0/1971381366 >> 172.21.15.7:0/1971381366 conn(0x55f1cd269a80 msgr2=0x55f1cd2b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 -- 172.21.15.7:0/1971381366 shutdown_connections 2024-09-15T23:36:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 1 -- 172.21.15.7:0/1971381366 wait complete. 2024-09-15T23:36:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.777+0000 7f460c674740 20 client.4407 trim_cache size 0 max 16384 2024-09-15T23:36:30.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.795+0000 7f460c674740 1 Processor -- start 2024-09-15T23:36:30.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.795+0000 7f460c674740 1 -- start start 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f460c674740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f460c674740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 0x55f1cd328e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f460c674740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd296890 0x55f1cd291f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f460c674740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4604201a20 con 0x55f1cd26c2c0 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f460c674740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4604200d80 con 0x55f1cd26da50 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f460c674740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f46042013d0 con 0x55f1cd296890 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f4602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f4602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53722/0 (socket says 172.21.15.7:53722) 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f4602ffd640 1 -- 172.21.15.7:0/3537421420 learned_addr learned my addr 172.21.15.7:0/3537421420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f46027fc640 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 0x55f1cd328e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f46037fe640 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd296890 0x55f1cd291f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f4602ffd640 1 -- 172.21.15.7:0/3537421420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd296890 msgr2=0x55f1cd291f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f4602ffd640 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd296890 0x55f1cd291f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f4602ffd640 1 -- 172.21.15.7:0/3537421420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 msgr2=0x55f1cd328e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f4602ffd640 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 0x55f1cd328e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f4602ffd640 1 -- 172.21.15.7:0/3537421420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1cd28df60 con 0x55f1cd26c2c0 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.796+0000 7f46037fe640 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd296890 0x55f1cd291f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f4602ffd640 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b2270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009ac0 tx=0x7f45f407b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d843ce69374ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f45e8ff9640 1 -- 172.21.15.7:0/3537421420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4077450 con 0x55f1cd26c2c0 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f45e8ff9640 1 -- 172.21.15.7:0/3537421420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f40776b0 con 0x55f1cd26c2c0 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f45e8ff9640 1 -- 172.21.15.7:0/3537421420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4075a20 con 0x55f1cd26c2c0 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f460c674740 1 -- 172.21.15.7:0/3537421420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 msgr2=0x55f1cd2b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f460c674740 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b2270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009ac0 tx=0x7f45f407b000 comp rx=0 tx=0).stop 2024-09-15T23:36:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f460c674740 1 -- 172.21.15.7:0/3537421420 shutdown_connections 2024-09-15T23:36:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f460c674740 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd296890 0x55f1cd291f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f460c674740 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 0x55f1cd328e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f460c674740 1 --2- 172.21.15.7:0/3537421420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd26c2c0 0x55f1cd2b2270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f460c674740 1 -- 172.21.15.7:0/3537421420 >> 172.21.15.7:0/3537421420 conn(0x55f1cd269a80 msgr2=0x55f1cd28d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f460c674740 1 -- 172.21.15.7:0/3537421420 shutdown_connections 2024-09-15T23:36:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.797+0000 7f460c674740 1 -- 172.21.15.7:0/3537421420 wait complete. 2024-09-15T23:36:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.798+0000 7f460c674740 1 -- start start 2024-09-15T23:36:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.798+0000 7f460c674740 1 Processor -- start 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f460c674740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 0x55f1cd327860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f460c674740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 0x55f1cd327da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f460c674740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd296890 0x55f1cd2c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f460c674740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f1cd295e90 con 0x55f1cd296890 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f460c674740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f1cd1763e0 con 0x55f1cd26da50 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f460c674740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x55f1cd296d60 con 0x55f1cd26c2c0 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f460c674740 10 client.0 _ll_register_callbacks cb 0x7f45f8017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd296890 0x55f1cd2c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd296890 0x55f1cd2c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53736/0 (socket says 172.21.15.7:53736) 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46037fe640 1 -- 172.21.15.7:0/1667071523 learned_addr learned my addr 172.21.15.7:0/1667071523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f4602ffd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 0x55f1cd327860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46027fc640 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 0x55f1cd327da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46037fe640 1 -- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 msgr2=0x55f1cd327860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46037fe640 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 0x55f1cd327860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46037fe640 1 -- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 msgr2=0x55f1cd327da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46037fe640 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 0x55f1cd327da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46037fe640 1 -- 172.21.15.7:0/1667071523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1cd28d700 con 0x55f1cd296890 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46027fc640 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 0x55f1cd327da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f46037fe640 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd296890 0x55f1cd2c4170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f460415ee20 tx=0x7f45fc001c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977ce91635644af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f45e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f45e27fc640 1 -- 172.21.15.7:0/1667071523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc01a020 con 0x55f1cd296890 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f45e27fc640 1 -- 172.21.15.7:0/1667071523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f45fc016660 con 0x55f1cd296890 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.799+0000 7f460c674740 1 -- 172.21.15.7:0/1667071523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f40776b0 con 0x55f1cd296890 2024-09-15T23:36:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f45e27fc640 1 -- 172.21.15.7:0/1667071523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f45fc024050 con 0x55f1cd296890 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f460c674740 10 client.4413 fetch_fsmap learned FSMap version 6 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f460c674740 1 -- 172.21.15.7:0/1667071523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55f1cd2c4fc0 con 0x55f1cd296890 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f45e27fc640 1 -- 172.21.15.7:0/1667071523 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f40776b0 con 0x55f1cd296890 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f45e27fc640 1 -- 172.21.15.7:0/1667071523 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7f45fc027020 con 0x55f1cd296890 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f460c674740 10 client.4413 fetch_fsmap finished waiting for FSMap version 6 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f460c674740 10 client.4413 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f460c674740 20 client.4413 populate_metadata read hostname 'smithi007' 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f460c674740 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55f1cd27aa40 0x55f1cd2af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f460c674740 4 client.4413 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f460c674740 1 -- 172.21.15.7:0/1667071523 --> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] -- command(tid 0: {"prefix": "status"}) -- 0x55f1cd1d5fa0 con 0x55f1cd27aa40 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f4602ffd640 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55f1cd27aa40 0x55f1cd2af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f45e27fc640 10 client.4413 ms_handle_connect on v2:172.21.15.18:6832/3035612480 2024-09-15T23:36:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.800+0000 7f4602ffd640 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55f1cd27aa40 0x55f1cd2af1c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c6e248e54b8dcd server_cookie=206bee0f1a0521e4 in_seq=0 out_seq=0 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f45e27fc640 1 -- 172.21.15.7:0/1667071523 <== mds.0 v2:172.21.15.18:6832/3035612480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55f1cd1d5fa0 con 0x55f1cd27aa40 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f45e27fc640 10 client.4413 handle_command_reply: tid=0 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 client.4413 shutdown 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 -- 172.21.15.7:0/1667071523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd296890 msgr2=0x55f1cd2c4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd296890 0x55f1cd2c4170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f460415ee20 tx=0x7f45fc001c80 comp rx=0 tx=0).stop 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 -- 172.21.15.7:0/1667071523 shutdown_connections 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55f1cd27aa40 0x55f1cd2af1c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f1cd296890 0x55f1cd2c4170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1cd26da50 0x55f1cd327da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 --2- 172.21.15.7:0/1667071523 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55f1cd26c2c0 0x55f1cd327860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f45e27fc640 0 client.4413 ms_handle_reset on v2:172.21.15.18:6832/3035612480 2024-09-15T23:36:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 -- 172.21.15.7:0/1667071523 >> 172.21.15.7:0/1667071523 conn(0x55f1cd269a80 msgr2=0x55f1cd261ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 -- 172.21.15.7:0/1667071523 shutdown_connections 2024-09-15T23:36:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 1 -- 172.21.15.7:0/1667071523 wait complete. 2024-09-15T23:36:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f460c674740 20 client.4413 trim_cache size 0 max 16384 2024-09-15T23:36:30.809 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:36:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9da25f4c-4673-468c-a40c-f01392dffc9b", 2024-09-15T23:36:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-15T23:36:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4291, 2024-09-15T23:36:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-15T23:36:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-15T23:36:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-15T23:36:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.2371803620000001, 2024-09-15T23:36:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 6, 2024-09-15T23:36:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 25, 2024-09-15T23:36:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 25, 2024-09-15T23:36:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.255991107 2024-09-15T23:36:30.810 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f46098dd640 1 -- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f45f006c2e0 msgr2=0x7f45f006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f46098dd640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f45f006c2e0 0x7f45f006e7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45f8001eb0 tx=0x7f45f8009040 comp rx=0 tx=0).stop 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f46098dd640 1 -- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 msgr2=0x7f46040b9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f46098dd640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f46040b9dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034040 tx=0x7f45f4004730 comp rx=0 tx=0).stop 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.801+0000 7f46098dd640 1 -- 172.21.15.7:0/1767824174 shutdown_connections 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.802+0000 7f46098dd640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f45f006c2e0 0x7f45f006e7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.802+0000 7f46098dd640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f46040b5090 0x7f46041f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.802+0000 7f46098dd640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4604153160 0x7f46041f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.802+0000 7f46098dd640 1 --2- 172.21.15.7:0/1767824174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ec003b70 0x7f46040b9dd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.802+0000 7f46098dd640 1 -- 172.21.15.7:0/1767824174 >> 172.21.15.7:0/1767824174 conn(0x7f460408bfd0 msgr2=0x7f46040d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:30.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.802+0000 7f46098dd640 1 -- 172.21.15.7:0/1767824174 shutdown_connections 2024-09-15T23:36:30.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.802+0000 7f46098dd640 1 -- 172.21.15.7:0/1767824174 wait complete. 2024-09-15T23:36:30.813 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T23:36:30.813 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-15T23:36:30.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-15T23:36:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.915+0000 7fca9736d640 1 Processor -- start 2024-09-15T23:36:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.915+0000 7fca9736d640 1 -- start start 2024-09-15T23:36:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9736d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca90158e60 0x7fca900b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9736d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca900d0d00 0x7fca9006efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9736d640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca9006dfa0 con 0x7fca900b31c0 2024-09-15T23:36:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9736d640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fca90159240 con 0x7fca900d0d00 2024-09-15T23:36:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9736d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fca9006e210 con 0x7fca90158e60 2024-09-15T23:36:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9590a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca900d0d00 0x7fca9006efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9590a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca900d0d00 0x7fca9006efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53750/0 (socket says 172.21.15.7:53750) 2024-09-15T23:36:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9590a640 1 -- 172.21.15.7:0/2078856490 learned_addr learned my addr 172.21.15.7:0/2078856490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca95109640 1 --2- 172.21.15.7:0/2078856490 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca90158e60 0x7fca900b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9590a640 1 -- 172.21.15.7:0/2078856490 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca90158e60 msgr2=0x7fca900b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9590a640 1 --2- 172.21.15.7:0/2078856490 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca90158e60 0x7fca900b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9590a640 1 -- 172.21.15.7:0/2078856490 >> v1:172.21.15.18:6789/0 conn(0x7fca900b31c0 legacy=0x7fca900d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.916+0000 7fca9590a640 1 -- 172.21.15.7:0/2078856490 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9006e4f0 con 0x7fca900d0d00 2024-09-15T23:36:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.917+0000 7fca9590a640 1 --2- 172.21.15.7:0/2078856490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 0x7fca9006efd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fca88009e50 tx=0x7fca8802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b915e66fb54aec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.917+0000 7fca87fff640 1 -- 172.21.15.7:0/2078856490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88036070 con 0x7fca900d0d00 2024-09-15T23:36:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.917+0000 7fca87fff640 1 -- 172.21.15.7:0/2078856490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca88031600 con 0x7fca900d0d00 2024-09-15T23:36:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.917+0000 7fca87fff640 1 -- 172.21.15.7:0/2078856490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca880319d0 con 0x7fca900d0d00 2024-09-15T23:36:30.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.918+0000 7fca9736d640 1 -- 172.21.15.7:0/2078856490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 msgr2=0x7fca9006efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.918+0000 7fca9736d640 1 --2- 172.21.15.7:0/2078856490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 0x7fca9006efd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fca88009e50 tx=0x7fca8802a7a0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.918+0000 7fca9736d640 1 -- 172.21.15.7:0/2078856490 shutdown_connections 2024-09-15T23:36:30.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.918+0000 7fca9736d640 1 --2- 172.21.15.7:0/2078856490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 0x7fca9006efd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.918+0000 7fca9736d640 1 --2- 172.21.15.7:0/2078856490 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca90158e60 0x7fca900b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.918+0000 7fca9736d640 1 -- 172.21.15.7:0/2078856490 >> 172.21.15.7:0/2078856490 conn(0x7fca900b21e0 msgr2=0x7fca900b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:30.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.918+0000 7fca9736d640 1 -- 172.21.15.7:0/2078856490 shutdown_connections 2024-09-15T23:36:30.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.918+0000 7fca9736d640 1 -- 172.21.15.7:0/2078856490 wait complete. 2024-09-15T23:36:30.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.919+0000 7fca9736d640 1 Processor -- start 2024-09-15T23:36:30.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.919+0000 7fca9736d640 1 -- start start 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca9736d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca900b31c0 0x7fca900ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca9736d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 0x7fca901574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca9736d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca90158e60 0x7fca90157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca9736d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca900b3bc0 con 0x7fca900d0d00 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca9736d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fca9006eab0 con 0x7fca90158e60 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca9736d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fca900b3d40 con 0x7fca900b31c0 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca94908640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 0x7fca901574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca95109640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca900b31c0 0x7fca900ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca95109640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca900b31c0 0x7fca900ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:52646/0 (socket says 172.21.15.7:52646) 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca94908640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 0x7fca901574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53752/0 (socket says 172.21.15.7:53752) 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca95109640 1 -- 172.21.15.7:0/3033267464 learned_addr learned my addr 172.21.15.7:0/3033267464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca9590a640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca90158e60 0x7fca90157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca95109640 1 -- 172.21.15.7:0/3033267464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca90158e60 msgr2=0x7fca90157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca95109640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca90158e60 0x7fca90157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca95109640 1 -- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 msgr2=0x7fca901574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca95109640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 0x7fca901574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca95109640 1 -- 172.21.15.7:0/3033267464 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca901f0c70 con 0x7fca900b31c0 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca94908640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 0x7fca901574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.920+0000 7fca95109640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca900b31c0 0x7fca900ada50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d5f0 tx=0x7fca8c00dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.921+0000 7fca85ffb640 1 -- 172.21.15.7:0/3033267464 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0125f0 con 0x7fca900b31c0 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.921+0000 7fca85ffb640 1 -- 172.21.15.7:0/3033267464 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca8c019070 con 0x7fca900b31c0 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.921+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca901f19f0 con 0x7fca900b31c0 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.921+0000 7fca85ffb640 1 -- 172.21.15.7:0/3033267464 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0129d0 con 0x7fca900b31c0 2024-09-15T23:36:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.921+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fca901f14d0 con 0x7fca900b31c0 2024-09-15T23:36:30.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.922+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9015ca20 con 0x7fca900b31c0 2024-09-15T23:36:30.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.922+0000 7fca85ffb640 1 -- 172.21.15.7:0/3033267464 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fca8c012b70 con 0x7fca900b31c0 2024-09-15T23:36:30.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.923+0000 7fca85ffb640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fca7406c2b0 0x7fca7406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:30.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.923+0000 7fca94908640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fca7406c2b0 0x7fca7406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:30.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.923+0000 7fca85ffb640 1 -- 172.21.15.7:0/3033267464 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fca8c08bb90 con 0x7fca900b31c0 2024-09-15T23:36:30.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.923+0000 7fca94908640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fca7406c2b0 0x7fca7406e770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca90158730 tx=0x7fca80007e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:30.928+0000 7fca85ffb640 1 -- 172.21.15.7:0/3033267464 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca8c059850 con 0x7fca900b31c0 2024-09-15T23:36:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.142+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fca901f0fc0 con 0x7fca900b31c0 2024-09-15T23:36:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.143+0000 7fca85ffb640 1 -- 172.21.15.7:0/3033267464 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7fca8c05db50 con 0x7fca900b31c0 2024-09-15T23:36:31.144 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T23:36:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fca7406c2b0 msgr2=0x7fca7406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fca7406c2b0 0x7fca7406e770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca90158730 tx=0x7fca80007e50 comp rx=0 tx=0).stop 2024-09-15T23:36:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca900b31c0 msgr2=0x7fca900ada50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca900b31c0 0x7fca900ada50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d5f0 tx=0x7fca8c00dab0 comp rx=0 tx=0).stop 2024-09-15T23:36:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 shutdown_connections 2024-09-15T23:36:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca90158e60 0x7fca90157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca900d0d00 0x7fca901574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fca7406c2b0 0x7fca7406e770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 --2- 172.21.15.7:0/3033267464 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca900b31c0 0x7fca900ada50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 >> 172.21.15.7:0/3033267464 conn(0x7fca900b21e0 msgr2=0x7fca9015b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 shutdown_connections 2024-09-15T23:36:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31.145+0000 7fca9736d640 1 -- 172.21.15.7:0/3033267464 wait complete. 2024-09-15T23:36:31.156 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-15T23:36:31.156 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T23:36:31.157 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-15T23:36:31.157 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-15T23:36:31.157 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-15T23:36:31.157 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-15T23:36:31.157 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T23:36:31.157 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T23:36:31.157 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-15T23:36:31.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:31.157 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-15T23:36:31.173 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:36:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:36:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:36:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-15T23:36:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:31.173 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42764sec preferred_lft 42764sec 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:31.174 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-15T23:36:31.175 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T23:36:31.175 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:31.175 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:31.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-09-15T23:36:31.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-09-15T23:36:31.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-09-15T23:36:31.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T23:36:31.176 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:31.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:31.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:31.407 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T23:36:31.433 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-15T23:36:31.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:31.435 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-15T23:36:31.508 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-15T23:36:31.508 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T23:36:31.508 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T23:36:31.508 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T23:36:31.508 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T23:36:31.509 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:31.509 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:31.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T23:36:31.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T23:36:31.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T23:36:31.509 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:31.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:31.825 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-15T23:36:31.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:31.842 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-09-15T23:36:31.898 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:31.898 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:31.898 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T23:36:31.898 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-15T23:36:31.898 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:32.018 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-15T23:36:32.018 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:32.018 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:32.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-15T23:36:32.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T23:36:32.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-15T23:36:32.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-15T23:36:32.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-15T23:36:32.019 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:32.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:32.154 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:32.154 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:32.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-15T23:36:32.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T23:36:32.155 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:32.235 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-15T23:36:32.235 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:32.235 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-15T23:36:32.252 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-15T23:36:32.253 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:32.253 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-15T23:36:32.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-15T23:36:32.375 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-09-15T23:36:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-09-15T23:36:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-09-15T23:36:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-09-15T23:36:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-09-15T23:36:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-09-15T23:36:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-09-15T23:36:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-09-15T23:36:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-09-15T23:36:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-09-15T23:36:32.433 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-0 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-1 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-2 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-3 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T23:36:32.434 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T23:36:32.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:32.435 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-15T23:36:32.491 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-15T23:36:32.491 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-15T23:36:32.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-15T23:36:32.559 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-09-15T23:36:32.622 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-09-15T23:36:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-09-15T23:36:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-0 2024-09-15T23:36:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-1 2024-09-15T23:36:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-2 2024-09-15T23:36:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-3 2024-09-15T23:36:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-09-15T23:36:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T23:36:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T23:36:32.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:32.624 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-15T23:36:32.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:2024-09-15T23:36:32.636+0000 7f3a9ac0f480 -1 init, newargv = 0x562eaf0096a0 newargc=15 2024-09-15T23:36:32.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[42942]: starting ceph client 2024-09-15T23:36:32.644 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[42942]: starting fuse 2024-09-15T23:36:32.654 INFO:teuthology.orchestra.run.smithi007.stdout:43 2024-09-15T23:36:32.654 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-09-15T23:36:32.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo stdin-killer -- python3 -c ' 2024-09-15T23:36:32.654 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-09-15T23:36:32.654 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-09-15T23:36:32.654 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-15T23:36:32.654 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-09-15T23:36:32.654 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-15T23:36:32.654 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-09-15T23:36:32.655 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-09-15T23:36:32.656 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-09-15T23:36:32.656 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-15T23:36:32.656 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-15T23:36:32.656 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.admin")) 2024-09-15T23:36:32.656 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-15T23:36:32.773 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.admin.42942.asok 2024-09-15T23:36:32.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:32.784 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42942.asok 2024-09-15T23:36:32.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:32.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42942.asok status 2024-09-15T23:36:32.922 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:36:32.922 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-09-15T23:36:32.922 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-15T23:36:32.922 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T23:36:32.922 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "admin", 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "42942", 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4428, 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4428 2024-09-15T23:36:32.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4064256669 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4064256669 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4428 192.168.0.1:0/4064256669", 2024-09-15T23:36:32.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/4064256669", 2024-09-15T23:36:32.925 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-09-15T23:36:32.925 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 6, 2024-09-15T23:36:32.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 25, 2024-09-15T23:36:32.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-09-15T23:36:32.925 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-09-15T23:36:32.925 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-09-15T23:36:32.925 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:36:32.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:32.931 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-09-15T23:36:32.931 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-15T23:36:32.947 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-09-15T23:36:32.948 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-15T23:36:32.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:32.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-15T23:36:33.014 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-15T23:36:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:33.127 INFO:teuthology.orchestra.run.smithi007.stdout:# file: . 2024-09-15T23:36:33.127 INFO:teuthology.orchestra.run.smithi007.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-15T23:36:33.127 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:33.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:33.133 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:36:33.133 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:36:33.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-15T23:36:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.293+0000 7fb1c014b640 1 Processor -- start 2024-09-15T23:36:33.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.293+0000 7fb1c014b640 1 -- start start 2024-09-15T23:36:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.294+0000 7fb1c014b640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb1b815e5b0 0x7fb1b8162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.294+0000 7fb1c014b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 0x7fb1b80cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.294+0000 7fb1c014b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b80d2980 con 0x7fb1b80d24e0 2024-09-15T23:36:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.294+0000 7fb1c014b640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b80d2c40 con 0x7fb1b815e5b0 2024-09-15T23:36:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.294+0000 7fb1c014b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b80d2f20 con 0x7fb1b815e990 2024-09-15T23:36:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.294+0000 7fb1bd6e6640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb1b80d24e0 0x7fb1b8162890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44748/0 (socket says 172.21.15.7:44748) 2024-09-15T23:36:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.294+0000 7fb1bd6e6640 1 -- 172.21.15.7:0/1607212452 learned_addr learned my addr 172.21.15.7:0/1607212452 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.295+0000 7fb1bdee7640 1 --2- 172.21.15.7:0/1607212452 >> v2:172.21.15.18:3300/0 conn(0x7fb1b815e5b0 0x7fb1b8162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.295+0000 7fb1be6e8640 1 --2- 172.21.15.7:0/1607212452 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 0x7fb1b80cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.295+0000 7fb1bcee5640 1 -- 172.21.15.7:0/1607212452 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464828710 0 0) 0x7fb1b80d2980 con 0x7fb1b80d24e0 2024-09-15T23:36:33.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.295+0000 7fb1bcee5640 1 -- 172.21.15.7:0/1607212452 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1a0003520 con 0x7fb1b80d24e0 2024-09-15T23:36:33.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.295+0000 7fb1bdee7640 1 -- 172.21.15.7:0/1607212452 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 msgr2=0x7fb1b80cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:33.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.295+0000 7fb1bdee7640 1 --2- 172.21.15.7:0/1607212452 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 0x7fb1b80cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.295+0000 7fb1bcee5640 1 -- 172.21.15.7:0/1607212452 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1275112873 0 0) 0x7fb1a0003520 con 0x7fb1b80d24e0 2024-09-15T23:36:33.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.295+0000 7fb1bdee7640 1 -- 172.21.15.7:0/1607212452 >> v1:172.21.15.7:6789/0 conn(0x7fb1b80d24e0 legacy=0x7fb1b8162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.295+0000 7fb1bdee7640 1 -- 172.21.15.7:0/1607212452 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b80d3200 con 0x7fb1b815e5b0 2024-09-15T23:36:33.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.296+0000 7fb1bdee7640 1 --2- 172.21.15.7:0/1607212452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 0x7fb1b8162350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009dc0 tx=0x7fb1a802a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ab4adef6d3c953f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:33.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.296+0000 7fb1bcee5640 1 -- 172.21.15.7:0/1607212452 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0016c0 con 0x7fb1b815e5b0 2024-09-15T23:36:33.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.296+0000 7fb1bcee5640 1 -- 172.21.15.7:0/1607212452 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a8007540 con 0x7fb1b815e5b0 2024-09-15T23:36:33.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.296+0000 7fb1bcee5640 1 -- 172.21.15.7:0/1607212452 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80078f0 con 0x7fb1b815e5b0 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.296+0000 7fb1c014b640 1 -- 172.21.15.7:0/1607212452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 msgr2=0x7fb1b8162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.296+0000 7fb1c014b640 1 --2- 172.21.15.7:0/1607212452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 0x7fb1b8162350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009dc0 tx=0x7fb1a802a610 comp rx=0 tx=0).stop 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.297+0000 7fb1c014b640 1 -- 172.21.15.7:0/1607212452 shutdown_connections 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.297+0000 7fb1c014b640 1 --2- 172.21.15.7:0/1607212452 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 0x7fb1b80cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.297+0000 7fb1c014b640 1 --2- 172.21.15.7:0/1607212452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 0x7fb1b8162350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.297+0000 7fb1c014b640 1 -- 172.21.15.7:0/1607212452 >> 172.21.15.7:0/1607212452 conn(0x7fb1b808bfd0 msgr2=0x7fb1b80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.297+0000 7fb1c014b640 1 -- 172.21.15.7:0/1607212452 shutdown_connections 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.297+0000 7fb1c014b640 1 -- 172.21.15.7:0/1607212452 wait complete. 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.298+0000 7fb1c014b640 1 Processor -- start 2024-09-15T23:36:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.298+0000 7fb1c014b640 1 -- start start 2024-09-15T23:36:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1c014b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b80d24e0 0x7fb1b80b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1c014b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 0x7fb1b80cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1c014b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 0x7fb1b81504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1c014b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80b3b40 con 0x7fb1b80d24e0 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1c014b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80b39c0 con 0x7fb1b815e5b0 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1c014b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b8162fa0 con 0x7fb1b815e990 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bdee7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b80d24e0 0x7fb1b80b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bdee7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b80d24e0 0x7fb1b80b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53758/0 (socket says 172.21.15.7:53758) 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bdee7640 1 -- 172.21.15.7:0/2829676573 learned_addr learned my addr 172.21.15.7:0/2829676573 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1be6e8640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 0x7fb1b81504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bd6e6640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 0x7fb1b80cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bdee7640 1 -- 172.21.15.7:0/2829676573 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 msgr2=0x7fb1b81504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bdee7640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 0x7fb1b81504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bdee7640 1 -- 172.21.15.7:0/2829676573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 msgr2=0x7fb1b80cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bdee7640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 0x7fb1b80cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bdee7640 1 -- 172.21.15.7:0/2829676573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b80b8020 con 0x7fb1b80d24e0 2024-09-15T23:36:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bd6e6640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 0x7fb1b80cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1bdee7640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b80d24e0 0x7fb1b80b7ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009ee0 tx=0x7fb1a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1b67fc640 1 -- 172.21.15.7:0/2829676573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8009450 con 0x7fb1b80d24e0 2024-09-15T23:36:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1b67fc640 1 -- 172.21.15.7:0/2829676573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a8004240 con 0x7fb1b80d24e0 2024-09-15T23:36:33.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b80ccce0 con 0x7fb1b80d24e0 2024-09-15T23:36:33.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.299+0000 7fb1b67fc640 1 -- 172.21.15.7:0/2829676573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8004630 con 0x7fb1b80d24e0 2024-09-15T23:36:33.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.300+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b80cd540 con 0x7fb1b80d24e0 2024-09-15T23:36:33.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.301+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b8162350 con 0x7fb1b80d24e0 2024-09-15T23:36:33.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.301+0000 7fb1b67fc640 1 -- 172.21.15.7:0/2829676573 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb1a8039040 con 0x7fb1b80d24e0 2024-09-15T23:36:33.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.302+0000 7fb1b67fc640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1a406c2b0 0x7fb1a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.302+0000 7fb1b67fc640 1 -- 172.21.15.7:0/2829676573 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb1a80accb0 con 0x7fb1b80d24e0 2024-09-15T23:36:33.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.302+0000 7fb1bd6e6640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1a406c2b0 0x7fb1a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.302+0000 7fb1bd6e6640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1a406c2b0 0x7fb1a406e770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac001eb0 tx=0x7fb1ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:33.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.306+0000 7fb1b67fc640 1 -- 172.21.15.7:0/2829676573 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb1a8040030 con 0x7fb1b80d24e0 2024-09-15T23:36:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.493+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1b80b3400 con 0x7fb1b80d24e0 2024-09-15T23:36:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.493+0000 7fb1b67fc640 1 -- 172.21.15.7:0/2829676573 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fb1a807b200 con 0x7fb1b80d24e0 2024-09-15T23:36:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-15T23:36:33.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1a406c2b0 msgr2=0x7fb1a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1a406c2b0 0x7fb1a406e770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac001eb0 tx=0x7fb1ac009040 comp rx=0 tx=0).stop 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b80d24e0 msgr2=0x7fb1b80b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b80d24e0 0x7fb1b80b7ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009ee0 tx=0x7fb1a8004a40 comp rx=0 tx=0).stop 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 shutdown_connections 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1a406c2b0 0x7fb1a406e770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1b815e990 0x7fb1b81504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1b815e5b0 0x7fb1b80cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 --2- 172.21.15.7:0/2829676573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b80d24e0 0x7fb1b80b7ae0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 >> 172.21.15.7:0/2829676573 conn(0x7fb1b808bfd0 msgr2=0x7fb1b806cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 shutdown_connections 2024-09-15T23:36:33.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.495+0000 7fb1c014b640 1 -- 172.21.15.7:0/2829676573 wait complete. 2024-09-15T23:36:33.507 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:36:33.507 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:36:33.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-15T23:36:33.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.666+0000 7f5a8f32a640 1 Processor -- start 2024-09-15T23:36:33.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.667+0000 7f5a8f32a640 1 -- start start 2024-09-15T23:36:33.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.667+0000 7f5a8f32a640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5a8815e5b0 0x7f5a88162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8f32a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a8815e990 0x7f5a880cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8f32a640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a880d2980 con 0x7f5a880d24e0 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8f32a640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f5a880d2c40 con 0x7f5a8815e5b0 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8f32a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5a880d2f20 con 0x7f5a8815e990 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8c8c5640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5a880d24e0 0x7f5a88162890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44750/0 (socket says 172.21.15.7:44750) 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8c8c5640 1 -- 172.21.15.7:0/3396328427 learned_addr learned my addr 172.21.15.7:0/3396328427 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8d8c7640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a8815e990 0x7f5a880cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8d0c6640 1 --2- 172.21.15.7:0/3396328427 >> v2:172.21.15.18:3300/0 conn(0x7f5a8815e5b0 0x7f5a88162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 877983292 0 0) 0x7f5a880d2980 con 0x7f5a880d24e0 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a74003520 con 0x7f5a880d24e0 2024-09-15T23:36:33.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2694309622 0 0) 0x7f5a74003520 con 0x7f5a880d24e0 2024-09-15T23:36:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a8815e990 msgr2=0x7f5a880cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a8815e990 0x7f5a880cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 >> v2:172.21.15.18:3300/0 conn(0x7f5a8815e5b0 msgr2=0x7f5a88162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 --2- 172.21.15.7:0/3396328427 >> v2:172.21.15.18:3300/0 conn(0x7f5a8815e5b0 0x7f5a88162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a880d3200 con 0x7f5a880d24e0 2024-09-15T23:36:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f5a780017f0 con 0x7f5a880d24e0 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 >> v1:172.21.15.7:6789/0 conn(0x7f5a880d24e0 legacy=0x7f5a88162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740039b0 0x7f5a74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a740043a0 0x7f5a74004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a74004ef0 0x7f5a7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a880d2c40 con 0x7f5a740043a0 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5a880d2f20 con 0x7f5a74004ef0 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7400bf00 con 0x7f5a740039b0 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8d0c6640 1 --2- 172.21.15.7:0/3396328427 >> v2:172.21.15.18:3300/0 conn(0x7f5a8815e5b0 0x7f5a88162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8d8c7640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a8815e990 0x7f5a880cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.668+0000 7f5a8c8c5640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a740043a0 0x7f5a74004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8d0c6640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740039b0 0x7f5a74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8d8c7640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a74004ef0 0x7f5a7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8c8c5640 1 -- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740039b0 msgr2=0x7f5a74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8c8c5640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740039b0 0x7f5a74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8c8c5640 1 -- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a74004ef0 msgr2=0x7f5a7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8c8c5640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a74004ef0 0x7f5a7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8c8c5640 1 -- 172.21.15.7:0/3396328427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a74003520 con 0x7f5a740043a0 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8d0c6640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740039b0 0x7f5a74003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8d8c7640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a74004ef0 0x7f5a7400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.669+0000 7f5a8c8c5640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a740043a0 0x7f5a74004880 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800c1c0 tx=0x7f5a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cc6e49898b3b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78007d00 con 0x7f5a740043a0 2024-09-15T23:36:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a78007270 con 0x7f5a740043a0 2024-09-15T23:36:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a7ffff640 1 -- 172.21.15.7:0/3396328427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a780057a0 con 0x7f5a740043a0 2024-09-15T23:36:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a8f32a640 1 -- 172.21.15.7:0/3396328427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a740043a0 msgr2=0x7f5a74004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a8f32a640 1 --2- 172.21.15.7:0/3396328427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a740043a0 0x7f5a74004880 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800c1c0 tx=0x7f5a74005830 comp rx=0 tx=0).stop 2024-09-15T23:36:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a8d8c7640 1 -- 172.21.15.7:0/3396328427 reap_dead start 2024-09-15T23:36:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a8f32a640 1 -- 172.21.15.7:0/3396328427 shutdown_connections 2024-09-15T23:36:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a8f32a640 1 -- 172.21.15.7:0/3396328427 >> 172.21.15.7:0/3396328427 conn(0x7f5a8808bfd0 msgr2=0x7f5a880d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a8f32a640 1 -- 172.21.15.7:0/3396328427 shutdown_connections 2024-09-15T23:36:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.670+0000 7f5a8f32a640 1 -- 172.21.15.7:0/3396328427 wait complete. 2024-09-15T23:36:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.671+0000 7f5a8f32a640 1 Processor -- start 2024-09-15T23:36:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8f32a640 1 -- start start 2024-09-15T23:36:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8f32a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740043a0 0x7f5a880b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8f32a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a8815a5b0 0x7f5a880b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8f32a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a880cd5d0 0x7f5a880cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8f32a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a880b3b40 con 0x7f5a8815a5b0 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8f32a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5a880b39c0 con 0x7f5a880cd5d0 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8f32a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5a88162fa0 con 0x7f5a740043a0 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8c8c5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a8815a5b0 0x7f5a880b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8c8c5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a8815a5b0 0x7f5a880b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53762/0 (socket says 172.21.15.7:53762) 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8c8c5640 1 -- 172.21.15.7:0/63147949 learned_addr learned my addr 172.21.15.7:0/63147949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.672+0000 7f5a8d0c6640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740043a0 0x7f5a880b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8d8c7640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a880cd5d0 0x7f5a880cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8c8c5640 1 -- 172.21.15.7:0/63147949 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740043a0 msgr2=0x7f5a880b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8c8c5640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740043a0 0x7f5a880b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8c8c5640 1 -- 172.21.15.7:0/63147949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a880cd5d0 msgr2=0x7f5a880cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8c8c5640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a880cd5d0 0x7f5a880cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8c8c5640 1 -- 172.21.15.7:0/63147949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a880ba120 con 0x7f5a8815a5b0 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8d0c6640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740043a0 0x7f5a880b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8c8c5640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a8815a5b0 0x7f5a880b5c20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a78034040 tx=0x7f5a880d1fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a7dffb640 1 -- 172.21.15.7:0/63147949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78041030 con 0x7f5a8815a5b0 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a7dffb640 1 -- 172.21.15.7:0/63147949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a78007070 con 0x7f5a8815a5b0 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a880ccb30 con 0x7f5a8815a5b0 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a7dffb640 1 -- 172.21.15.7:0/63147949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78005bb0 con 0x7f5a8815a5b0 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.673+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a880ccdd0 con 0x7f5a8815a5b0 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.674+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a88162350 con 0x7f5a8815a5b0 2024-09-15T23:36:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.675+0000 7f5a7dffb640 1 -- 172.21.15.7:0/63147949 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5a78044030 con 0x7f5a8815a5b0 2024-09-15T23:36:33.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.675+0000 7f5a7dffb640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a6c06c2b0 0x7f5a6c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:33.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.676+0000 7f5a7dffb640 1 -- 172.21.15.7:0/63147949 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5a780abb20 con 0x7f5a8815a5b0 2024-09-15T23:36:33.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.676+0000 7f5a8d0c6640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a6c06c2b0 0x7f5a6c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:33.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.676+0000 7f5a8d0c6640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a6c06c2b0 0x7f5a6c06e770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a881f7e60 tx=0x7f5a84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:33.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.680+0000 7f5a7dffb640 1 -- 172.21.15.7:0/63147949 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5a7804b070 con 0x7f5a8815a5b0 2024-09-15T23:36:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.869+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a880ba560 con 0x7f5a8815a5b0 2024-09-15T23:36:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.869+0000 7f5a7dffb640 1 -- 172.21.15.7:0/63147949 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f5a78079fa0 con 0x7f5a8815a5b0 2024-09-15T23:36:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.871+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a6c06c2b0 msgr2=0x7f5a6c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.871+0000 7f5a8f32a640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a6c06c2b0 0x7f5a6c06e770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a881f7e60 tx=0x7f5a84008040 comp rx=0 tx=0).stop 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.871+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a8815a5b0 msgr2=0x7f5a880b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.871+0000 7f5a8f32a640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a8815a5b0 0x7f5a880b5c20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a78034040 tx=0x7f5a880d1fb0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.872+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 shutdown_connections 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.872+0000 7f5a8f32a640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a880cd5d0 0x7f5a880cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.872+0000 7f5a8f32a640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a8815a5b0 0x7f5a880b5c20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.872+0000 7f5a8f32a640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a6c06c2b0 0x7f5a6c06e770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.872+0000 7f5a8f32a640 1 --2- 172.21.15.7:0/63147949 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a740043a0 0x7f5a880b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.872+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 >> 172.21.15.7:0/63147949 conn(0x7f5a8808bfd0 msgr2=0x7f5a880cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.872+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 shutdown_connections 2024-09-15T23:36:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:33.872+0000 7f5a8f32a640 1 -- 172.21.15.7:0/63147949 wait complete. 2024-09-15T23:36:33.885 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-09-15T23:36:33.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:33.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-15T23:36:33.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[42942]: fuse finished with error 0 and tester_r 0 2024-09-15T23:36:33.910 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:36:40.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:36:40.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:40.013 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-15T23:36:40.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:40.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.210+0000 7f4f4fcd0640 1 Processor -- start 2024-09-15T23:36:40.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.210+0000 7f4f4fcd0640 1 -- start start 2024-09-15T23:36:40.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.211+0000 7f4f4fcd0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f480d32a0 0x7f4f48169280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.211+0000 7f4f4fcd0640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f480b3d10 con 0x7f4f4815e6d0 2024-09-15T23:36:40.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.211+0000 7f4f4fcd0640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f480cfcd0 con 0x7f4f480d51c0 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.211+0000 7f4f4fcd0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f4f480d28c0 con 0x7f4f480d32a0 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.211+0000 7f4f4da6c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4f4815e6d0 0x7f4f480d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44598/0 (socket says 172.21.15.7:44598) 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.211+0000 7f4f4da6c640 1 -- 172.21.15.7:0/1787057768 learned_addr learned my addr 172.21.15.7:0/1787057768 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.211+0000 7f4f4e26d640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f480d32a0 0x7f4f48169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.211+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3112541195 0 0) 0x7f4f480b3d10 con 0x7f4f4815e6d0 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.211+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f34003520 con 0x7f4f4815e6d0 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 535478526 0 0) 0x7f4f34003520 con 0x7f4f4815e6d0 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f480d32a0 msgr2=0x7f4f48169280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f480d32a0 0x7f4f48169280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 >> v1:172.21.15.18:6789/0 conn(0x7f4f480d51c0 legacy=0x7f4f480d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f480cff50 con 0x7f4f4815e6d0 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f4f44001670 con 0x7f4f4815e6d0 2024-09-15T23:36:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 >> v1:172.21.15.7:6789/0 conn(0x7f4f4815e6d0 legacy=0x7f4f480d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f340039e0 0x7f4f34003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f480d32a0 0x7f4f340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f480d28c0 con 0x7f4f340039e0 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3400baa0 con 0x7f4f34004800 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3400bca0 con 0x7f4f480d32a0 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4da6c640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f340039e0 0x7f4f34003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4d26b640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4e26d640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f480d32a0 0x7f4f340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4da6c640 1 -- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f480d32a0 msgr2=0x7f4f340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4da6c640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f480d32a0 0x7f4f340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4da6c640 1 -- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f34004800 msgr2=0x7f4f34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4da6c640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.212+0000 7f4f4da6c640 1 -- 172.21.15.7:0/1787057768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f34003520 con 0x7f4f340039e0 2024-09-15T23:36:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.213+0000 7f4f4da6c640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f340039e0 0x7f4f34003e00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c210 tx=0x7f4f34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bded8bdb7605d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.213+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44007d10 con 0x7f4f340039e0 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.213+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f44005ad0 con 0x7f4f340039e0 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.213+0000 7f4f4ca6a640 1 -- 172.21.15.7:0/1787057768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44007070 con 0x7f4f340039e0 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.214+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/1787057768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f340039e0 msgr2=0x7f4f34003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.214+0000 7f4f4fcd0640 1 --2- 172.21.15.7:0/1787057768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f340039e0 0x7f4f34003e00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c210 tx=0x7f4f34005830 comp rx=0 tx=0).stop 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.214+0000 7f4f4e26d640 1 -- 172.21.15.7:0/1787057768 reap_dead start 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.214+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/1787057768 shutdown_connections 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.214+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/1787057768 >> 172.21.15.7:0/1787057768 conn(0x7f4f4808bfd0 msgr2=0x7f4f480d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.214+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/1787057768 shutdown_connections 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.214+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/1787057768 wait complete. 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.215+0000 7f4f4fcd0640 1 Processor -- start 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.215+0000 7f4f4fcd0640 1 -- start start 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4fcd0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f340039e0 0x7f4f480b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4fcd0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f480b8880 0x7f4f480b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4fcd0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f481e96f0 0x7f4f4815d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4fcd0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f480d0510 con 0x7f4f480b8880 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4fcd0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4815eab0 con 0x7f4f340039e0 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4fcd0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f4f480b51d0 con 0x7f4f481e96f0 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4d26b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f480b8880 0x7f4f480b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4d26b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f480b8880 0x7f4f480b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47974/0 (socket says 172.21.15.7:47974) 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4d26b640 1 -- 172.21.15.7:0/188512023 learned_addr learned my addr 172.21.15.7:0/188512023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4da6c640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f340039e0 0x7f4f480b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4e26d640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f481e96f0 0x7f4f4815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4d26b640 1 -- 172.21.15.7:0/188512023 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f481e96f0 msgr2=0x7f4f4815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4d26b640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f481e96f0 0x7f4f4815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4d26b640 1 -- 172.21.15.7:0/188512023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f340039e0 msgr2=0x7f4f480b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4d26b640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f340039e0 0x7f4f480b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4d26b640 1 -- 172.21.15.7:0/188512023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f480cd800 con 0x7f4f480b8880 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.216+0000 7f4f4d26b640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f480b8880 0x7f4f480b2c20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800f5f0 tx=0x7f4f3800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.217+0000 7f4f3e7fc640 1 -- 172.21.15.7:0/188512023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380052e0 con 0x7f4f480b8880 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.217+0000 7f4f3e7fc640 1 -- 172.21.15.7:0/188512023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f38018070 con 0x7f4f480b8880 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.217+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f480b7d10 con 0x7f4f480b8880 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.217+0000 7f4f3e7fc640 1 -- 172.21.15.7:0/188512023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38003810 con 0x7f4f480b8880 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.217+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f480b82d0 con 0x7f4f480b8880 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.218+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f480b8670 con 0x7f4f480b8880 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.218+0000 7f4f3e7fc640 1 -- 172.21.15.7:0/188512023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4f380039b0 con 0x7f4f480b8880 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.219+0000 7f4f3e7fc640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4f2c06c2b0 0x7f4f2c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.219+0000 7f4f4da6c640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4f2c06c2b0 0x7f4f2c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.219+0000 7f4f3e7fc640 1 -- 172.21.15.7:0/188512023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4f3808b3a0 con 0x7f4f480b8880 2024-09-15T23:36:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.220+0000 7f4f4da6c640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4f2c06c2b0 0x7f4f2c06e770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f44032000 tx=0x7f4f4402d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.224+0000 7f4f3e7fc640 1 -- 172.21.15.7:0/188512023 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f480cfcd0 con 0x7f4f480b8880 2024-09-15T23:36:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.402+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f480d2b90 con 0x7f4f480b8880 2024-09-15T23:36:40.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.403+0000 7f4f3e7fc640 1 -- 172.21.15.7:0/188512023 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7f4f38059210 con 0x7f4f480b8880 2024-09-15T23:36:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:40.404 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4f2c06c2b0 msgr2=0x7f4f2c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4f2c06c2b0 0x7f4f2c06e770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f44032000 tx=0x7f4f4402d000 comp rx=0 tx=0).stop 2024-09-15T23:36:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f480b8880 msgr2=0x7f4f480b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f480b8880 0x7f4f480b2c20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800f5f0 tx=0x7f4f3800fab0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 shutdown_connections 2024-09-15T23:36:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4f2c06c2b0 0x7f4f2c06e770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f481e96f0 0x7f4f4815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f480b8880 0x7f4f480b2c20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 --2- 172.21.15.7:0/188512023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f340039e0 0x7f4f480b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 >> 172.21.15.7:0/188512023 conn(0x7f4f4808bfd0 msgr2=0x7f4f480b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 shutdown_connections 2024-09-15T23:36:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.405+0000 7f4f4fcd0640 1 -- 172.21.15.7:0/188512023 wait complete. 2024-09-15T23:36:40.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.519+0000 7ff9b71e4640 1 Processor -- start 2024-09-15T23:36:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.519+0000 7ff9b71e4640 1 -- start start 2024-09-15T23:36:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.519+0000 7ff9b71e4640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9b015e840 0x7ff9b00d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.519+0000 7ff9b71e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 0x7ff9b006d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.519+0000 7ff9b71e4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff9b006dbb0 0x7ff9b00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b71e4640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b00cfe90 con 0x7ff9b015e840 2024-09-15T23:36:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b71e4640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b00cf560 con 0x7ff9b006dbb0 2024-09-15T23:36:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b71e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b00cf840 con 0x7ff9b00d52e0 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b4f80640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9b015e840 0x7ff9b00d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b4f80640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9b015e840 0x7ff9b00d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47982/0 (socket says 172.21.15.7:47982) 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b4f80640 1 -- 172.21.15.7:0/724801760 learned_addr learned my addr 172.21.15.7:0/724801760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b5781640 1 --2- 172.21.15.7:0/724801760 >> v2:172.21.15.18:3300/0 conn(0x7ff9b006dbb0 0x7ff9b00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9affff640 1 --2- 172.21.15.7:0/724801760 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 0x7ff9b006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b4f80640 1 -- 172.21.15.7:0/724801760 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 msgr2=0x7ff9b006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b4f80640 1 --2- 172.21.15.7:0/724801760 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 0x7ff9b006d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b4f80640 1 -- 172.21.15.7:0/724801760 >> v2:172.21.15.18:3300/0 conn(0x7ff9b006dbb0 msgr2=0x7ff9b00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b4f80640 1 --2- 172.21.15.7:0/724801760 >> v2:172.21.15.18:3300/0 conn(0x7ff9b006dbb0 0x7ff9b00d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.520+0000 7ff9b4f80640 1 -- 172.21.15.7:0/724801760 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b00cfb20 con 0x7ff9b015e840 2024-09-15T23:36:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b4f80640 1 --2- 172.21.15.7:0/724801760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 0x7ff9b00d0320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff9b008e8e0 tx=0x7ff9a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d10339db11a2bbb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9af7fe640 1 -- 172.21.15.7:0/724801760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0037070 con 0x7ff9b015e840 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9af7fe640 1 -- 172.21.15.7:0/724801760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a0032440 con 0x7ff9b015e840 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9af7fe640 1 -- 172.21.15.7:0/724801760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0032810 con 0x7ff9b015e840 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b71e4640 1 -- 172.21.15.7:0/724801760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 msgr2=0x7ff9b00d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/724801760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 0x7ff9b00d0320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff9b008e8e0 tx=0x7ff9a002a1e0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b71e4640 1 -- 172.21.15.7:0/724801760 shutdown_connections 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/724801760 >> v2:172.21.15.18:3300/0 conn(0x7ff9b006dbb0 0x7ff9b00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/724801760 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 0x7ff9b006d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/724801760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 0x7ff9b00d0320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b71e4640 1 -- 172.21.15.7:0/724801760 >> 172.21.15.7:0/724801760 conn(0x7ff9b008bfd0 msgr2=0x7ff9b00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b71e4640 1 -- 172.21.15.7:0/724801760 shutdown_connections 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.521+0000 7ff9b71e4640 1 -- 172.21.15.7:0/724801760 wait complete. 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.523+0000 7ff9b71e4640 1 Processor -- start 2024-09-15T23:36:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.523+0000 7ff9b71e4640 1 -- start start 2024-09-15T23:36:40.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b71e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9b006dbb0 0x7ff9b014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b71e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 0x7ff9b00cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b71e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 0x7ff9b00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b71e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b015ec20 con 0x7ff9b015e840 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b71e4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b00d0860 con 0x7ff9b006dbb0 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b71e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b00cb190 con 0x7ff9b00d52e0 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b5781640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 0x7ff9b00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b5781640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 0x7ff9b00cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47996/0 (socket says 172.21.15.7:47996) 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b5781640 1 -- 172.21.15.7:0/4039666934 learned_addr learned my addr 172.21.15.7:0/4039666934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b4f80640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9b006dbb0 0x7ff9b014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9affff640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 0x7ff9b00cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b5781640 1 -- 172.21.15.7:0/4039666934 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 msgr2=0x7ff9b00cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b5781640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 0x7ff9b00cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b5781640 1 -- 172.21.15.7:0/4039666934 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9b006dbb0 msgr2=0x7ff9b014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b5781640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9b006dbb0 0x7ff9b014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b5781640 1 -- 172.21.15.7:0/4039666934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b00cdda0 con 0x7ff9b015e840 2024-09-15T23:36:40.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.524+0000 7ff9b5781640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 0x7ff9b00cd570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8007cc0 tx=0x7ff9a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:40.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.525+0000 7ff9ad7fa640 1 -- 172.21.15.7:0/4039666934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8014420 con 0x7ff9b015e840 2024-09-15T23:36:40.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.525+0000 7ff9ad7fa640 1 -- 172.21.15.7:0/4039666934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a8019070 con 0x7ff9b015e840 2024-09-15T23:36:40.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.525+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b00ce080 con 0x7ff9b015e840 2024-09-15T23:36:40.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.525+0000 7ff9ad7fa640 1 -- 172.21.15.7:0/4039666934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8014800 con 0x7ff9b015e840 2024-09-15T23:36:40.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.525+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b00ce640 con 0x7ff9b015e840 2024-09-15T23:36:40.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.525+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b00d1cb0 con 0x7ff9b015e840 2024-09-15T23:36:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.526+0000 7ff9ad7fa640 1 -- 172.21.15.7:0/4039666934 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff9a80149a0 con 0x7ff9b015e840 2024-09-15T23:36:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.526+0000 7ff9ad7fa640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff99406c2b0 0x7ff99406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.527+0000 7ff9b4f80640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff99406c2b0 0x7ff99406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.527+0000 7ff9ad7fa640 1 -- 172.21.15.7:0/4039666934 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff9a808bd40 con 0x7ff9b015e840 2024-09-15T23:36:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.527+0000 7ff9b4f80640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff99406c2b0 0x7ff99406e770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0034000 tx=0x7ff9a002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:40.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.531+0000 7ff9ad7fa640 1 -- 172.21.15.7:0/4039666934 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff9a8059bb0 con 0x7ff9b015e840 2024-09-15T23:36:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.708+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9b00ce9b0 con 0x7ff9b015e840 2024-09-15T23:36:40.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.709+0000 7ff9ad7fa640 1 -- 172.21.15.7:0/4039666934 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7ff9a805deb0 con 0x7ff9b015e840 2024-09-15T23:36:40.710 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:40.710 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff99406c2b0 msgr2=0x7ff99406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff99406c2b0 0x7ff99406e770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0034000 tx=0x7ff9a002e000 comp rx=0 tx=0).stop 2024-09-15T23:36:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 msgr2=0x7ff9b00cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 0x7ff9b00cd570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8007cc0 tx=0x7ff9a800e4b0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 shutdown_connections 2024-09-15T23:36:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b015e840 0x7ff9b00cd570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff99406c2b0 0x7ff99406e770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9b00d52e0 0x7ff9b00cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 --2- 172.21.15.7:0/4039666934 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9b006dbb0 0x7ff9b014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 >> 172.21.15.7:0/4039666934 conn(0x7ff9b008bfd0 msgr2=0x7ff9b006cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 shutdown_connections 2024-09-15T23:36:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.711+0000 7ff9b71e4640 1 -- 172.21.15.7:0/4039666934 wait complete. 2024-09-15T23:36:40.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-15T23:36:40.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.827+0000 7f89098d0640 1 Processor -- start 2024-09-15T23:36:40.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.827+0000 7f89098d0640 1 -- start start 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.828+0000 7f89098d0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f890415f760 0x7f890406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.828+0000 7f89098d0640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f890415fb40 0x7f89041609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.828+0000 7f89098d0640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f890406c1f0 con 0x7f89040cce30 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.828+0000 7f89098d0640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f89040b4310 con 0x7f890415fb40 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.828+0000 7f89098d0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f890406c4d0 con 0x7f890415f760 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.828+0000 7f89027fc640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f890415fb40 0x7f89041609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.828+0000 7f89027fc640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f890415fb40 0x7f89041609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48008/0 (socket says 172.21.15.7:48008) 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.828+0000 7f89027fc640 1 -- 172.21.15.7:0/2999407380 learned_addr learned my addr 172.21.15.7:0/2999407380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.829+0000 7f8902ffd640 1 --2- 172.21.15.7:0/2999407380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f890415f760 0x7f890406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.829+0000 7f89027fc640 1 -- 172.21.15.7:0/2999407380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f890415f760 msgr2=0x7f890406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.829+0000 7f89027fc640 1 --2- 172.21.15.7:0/2999407380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f890415f760 0x7f890406bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.829+0000 7f89027fc640 1 -- 172.21.15.7:0/2999407380 >> v1:172.21.15.18:6789/0 conn(0x7f89040cce30 legacy=0x7f89040cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.829+0000 7f89027fc640 1 -- 172.21.15.7:0/2999407380 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890406c7b0 con 0x7f890415fb40 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.829+0000 7f8902ffd640 1 --2- 172.21.15.7:0/2999407380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f890415f760 0x7f890406bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.829+0000 7f89027fc640 1 --2- 172.21.15.7:0/2999407380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415fb40 0x7f89041609c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009870 tx=0x7f88ec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0fe91936e22dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f8901ffb640 1 -- 172.21.15.7:0/2999407380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec037070 con 0x7f890415fb40 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f8901ffb640 1 -- 172.21.15.7:0/2999407380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec032440 con 0x7f890415fb40 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f8901ffb640 1 -- 172.21.15.7:0/2999407380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec032810 con 0x7f890415fb40 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f89098d0640 1 -- 172.21.15.7:0/2999407380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415fb40 msgr2=0x7f89041609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f89098d0640 1 --2- 172.21.15.7:0/2999407380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415fb40 0x7f89041609c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009870 tx=0x7f88ec02a210 comp rx=0 tx=0).stop 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f89098d0640 1 -- 172.21.15.7:0/2999407380 shutdown_connections 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f89098d0640 1 --2- 172.21.15.7:0/2999407380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415fb40 0x7f89041609c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f89098d0640 1 --2- 172.21.15.7:0/2999407380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f890415f760 0x7f890406bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f89098d0640 1 -- 172.21.15.7:0/2999407380 >> 172.21.15.7:0/2999407380 conn(0x7f890409aa40 msgr2=0x7f89040b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:40.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f89098d0640 1 -- 172.21.15.7:0/2999407380 shutdown_connections 2024-09-15T23:36:40.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.830+0000 7f89098d0640 1 -- 172.21.15.7:0/2999407380 wait complete. 2024-09-15T23:36:40.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.831+0000 7f89098d0640 1 Processor -- start 2024-09-15T23:36:40.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.832+0000 7f89098d0640 1 -- start start 2024-09-15T23:36:40.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.832+0000 7f89098d0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89040cce30 0x7f8904158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.832+0000 7f89098d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415f760 0x7f890415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.832+0000 7f89098d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f890415fb40 0x7f890415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.832+0000 7f89098d0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89040b4920 con 0x7f890415f760 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.832+0000 7f89098d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f89040b47a0 con 0x7f890415fb40 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.832+0000 7f89098d0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f89040b4aa0 con 0x7f89040cce30 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415f760 0x7f890415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415f760 0x7f890415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48022/0 (socket says 172.21.15.7:48022) 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89027fc640 1 -- 172.21.15.7:0/387805390 learned_addr learned my addr 172.21.15.7:0/387805390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89037fe640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f890415fb40 0x7f890415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f8902ffd640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89040cce30 0x7f8904158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89027fc640 1 -- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89040cce30 msgr2=0x7f8904158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89027fc640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89040cce30 0x7f8904158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89027fc640 1 -- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f890415fb40 msgr2=0x7f890415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89027fc640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f890415fb40 0x7f890415bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89027fc640 1 -- 172.21.15.7:0/387805390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89040c74b0 con 0x7f890415f760 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f8902ffd640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89040cce30 0x7f8904158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89037fe640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f890415fb40 0x7f890415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89027fc640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415f760 0x7f890415b6b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0090b0 tx=0x7f88ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f88fbfff640 1 -- 172.21.15.7:0/387805390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec041020 con 0x7f890415f760 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f88fbfff640 1 -- 172.21.15.7:0/387805390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec030990 con 0x7f890415f760 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89040c7790 con 0x7f890415f760 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f88fbfff640 1 -- 172.21.15.7:0/387805390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec037040 con 0x7f890415f760 2024-09-15T23:36:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.833+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89040c7d50 con 0x7f890415f760 2024-09-15T23:36:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.834+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8904068630 con 0x7f890415f760 2024-09-15T23:36:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.835+0000 7f88fbfff640 1 -- 172.21.15.7:0/387805390 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f88ec044020 con 0x7f890415f760 2024-09-15T23:36:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.835+0000 7f88fbfff640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f88e406c2e0 0x7f88e406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.835+0000 7f88fbfff640 1 -- 172.21.15.7:0/387805390 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f88ec0ac140 con 0x7f890415f760 2024-09-15T23:36:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.835+0000 7f8902ffd640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f88e406c2e0 0x7f88e406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.836+0000 7f8902ffd640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f88e406c2e0 0x7f88e406e7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8904159280 tx=0x7f88f4007550 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:40.840+0000 7f88fbfff640 1 -- 172.21.15.7:0/387805390 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f88ec04b030 con 0x7f890415f760 2024-09-15T23:36:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.052+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f890406bc10 con 0x7f890415f760 2024-09-15T23:36:41.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.877+0000 7f88fbfff640 1 -- 172.21.15.7:0/387805390 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f88ec07a570 con 0x7f890415f760 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.879+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f88e406c2e0 msgr2=0x7f88e406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.879+0000 7f89098d0640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f88e406c2e0 0x7f88e406e7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8904159280 tx=0x7f88f4007550 comp rx=0 tx=0).stop 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.879+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415f760 msgr2=0x7f890415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.879+0000 7f89098d0640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415f760 0x7f890415b6b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0090b0 tx=0x7f88ec0046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.880+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 shutdown_connections 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.880+0000 7f89098d0640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f890415fb40 0x7f890415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.880+0000 7f89098d0640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f890415f760 0x7f890415b6b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.880+0000 7f89098d0640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f88e406c2e0 0x7f88e406e7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.880+0000 7f89098d0640 1 --2- 172.21.15.7:0/387805390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89040cce30 0x7f8904158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.880+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 >> 172.21.15.7:0/387805390 conn(0x7f890409aa40 msgr2=0x7f89040b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.880+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 shutdown_connections 2024-09-15T23:36:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:41.880+0000 7f89098d0640 1 -- 172.21.15.7:0/387805390 wait complete. 2024-09-15T23:36:41.895 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-15T23:36:41.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-15T23:36:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.006+0000 7f023e511640 1 Processor -- start 2024-09-15T23:36:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.006+0000 7f023e511640 1 -- start start 2024-09-15T23:36:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f023e511640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f02380d24e0 0x7f0238162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f023e511640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f023815e990 0x7f02380cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f023e511640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02380d2980 con 0x7f023815e5b0 2024-09-15T23:36:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f023e511640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f02380d2c40 con 0x7f023815e990 2024-09-15T23:36:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f023e511640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f02380d2f20 con 0x7f02380d24e0 2024-09-15T23:36:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0237fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f023815e5b0 0x7f0238162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44606/0 (socket says 172.21.15.7:44606) 2024-09-15T23:36:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0237fff640 1 -- 172.21.15.7:0/2049285534 learned_addr learned my addr 172.21.15.7:0/2049285534 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f02377fe640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f02380d24e0 0x7f0238162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f023caae640 1 --2- 172.21.15.7:0/2049285534 >> v2:172.21.15.18:3300/0 conn(0x7f023815e990 0x7f02380cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411230239 0 0) 0x7f02380d2980 con 0x7f023815e5b0 2024-09-15T23:36:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0220003520 con 0x7f023815e5b0 2024-09-15T23:36:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3414022308 0 0) 0x7f0220003520 con 0x7f023815e5b0 2024-09-15T23:36:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f02380d24e0 msgr2=0x7f0238162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f02380d24e0 0x7f0238162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 >> v2:172.21.15.18:3300/0 conn(0x7f023815e990 msgr2=0x7f02380cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:42.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 --2- 172.21.15.7:0/2049285534 >> v2:172.21.15.18:3300/0 conn(0x7f023815e990 0x7f02380cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02380d3200 con 0x7f023815e5b0 2024-09-15T23:36:42.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f0228001670 con 0x7f023815e5b0 2024-09-15T23:36:42.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f023caae640 1 --2- 172.21.15.7:0/2049285534 >> v2:172.21.15.18:3300/0 conn(0x7f023815e990 0x7f02380cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 >> v1:172.21.15.7:6789/0 conn(0x7f023815e5b0 legacy=0x7f0238162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02200039b0 0x7f0220003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0220004800 0x7f0220004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f02380d24e0 0x7f02200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02380d2c40 con 0x7f0220004800 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f02380d2f20 con 0x7f02200039b0 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.007+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f022000b9c0 con 0x7f02380d24e0 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.008+0000 7f02377fe640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0220004800 0x7f0220004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.008+0000 7f0237fff640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02200039b0 0x7f0220003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.008+0000 7f023caae640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f02380d24e0 0x7f02200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.008+0000 7f02377fe640 1 -- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f02380d24e0 msgr2=0x7f02200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.008+0000 7f02377fe640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f02380d24e0 0x7f02200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.008+0000 7f02377fe640 1 -- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02200039b0 msgr2=0x7f0220003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.008+0000 7f02377fe640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02200039b0 0x7f0220003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.008+0000 7f02377fe640 1 -- 172.21.15.7:0/2049285534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0220003520 con 0x7f0220004800 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.008+0000 7f0237fff640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02200039b0 0x7f0220003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f02377fe640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0220004800 0x7f0220004c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f022c009ae0 tx=0x7f0220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efd6d7f244ea2e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c036070 con 0x7f0220004800 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022c005990 con 0x7f0220004800 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f0236ffd640 1 -- 172.21.15.7:0/2049285534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c009680 con 0x7f0220004800 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f023e511640 1 -- 172.21.15.7:0/2049285534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0220004800 msgr2=0x7f0220004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f023e511640 1 --2- 172.21.15.7:0/2049285534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0220004800 0x7f0220004c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f022c009ae0 tx=0x7f0220005830 comp rx=0 tx=0).stop 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f023caae640 1 -- 172.21.15.7:0/2049285534 reap_dead start 2024-09-15T23:36:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f023e511640 1 -- 172.21.15.7:0/2049285534 shutdown_connections 2024-09-15T23:36:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f023e511640 1 -- 172.21.15.7:0/2049285534 >> 172.21.15.7:0/2049285534 conn(0x7f023808bfd0 msgr2=0x7f02380d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f023e511640 1 -- 172.21.15.7:0/2049285534 shutdown_connections 2024-09-15T23:36:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.009+0000 7f023e511640 1 -- 172.21.15.7:0/2049285534 wait complete. 2024-09-15T23:36:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.011+0000 7f023e511640 1 Processor -- start 2024-09-15T23:36:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.011+0000 7f023e511640 1 -- start start 2024-09-15T23:36:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.011+0000 7f023e511640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0220004800 0x7f0238161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.011+0000 7f023e511640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02381f4390 0x7f02381f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f023e511640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02381f4d50 0x7f023815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f023e511640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02380b3b40 con 0x7f02381f4390 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f023e511640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f02380b39c0 con 0x7f02381f4d50 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f023e511640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f0238162a60 con 0x7f0220004800 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f02377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02381f4390 0x7f02381f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f02377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02381f4390 0x7f02381f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48030/0 (socket says 172.21.15.7:48030) 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f02377fe640 1 -- 172.21.15.7:0/1881264772 learned_addr learned my addr 172.21.15.7:0/1881264772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f023caae640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02381f4d50 0x7f023815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f02377fe640 1 -- 172.21.15.7:0/1881264772 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0220004800 msgr2=0x7f0238161940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f0237fff640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0220004800 0x7f0238161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f02377fe640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0220004800 0x7f0238161940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f02377fe640 1 -- 172.21.15.7:0/1881264772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02381f4d50 msgr2=0x7f023815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f02377fe640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02381f4d50 0x7f023815f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f02377fe640 1 -- 172.21.15.7:0/1881264772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02380b4130 con 0x7f02381f4390 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f02377fe640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02381f4390 0x7f02381f4810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f022c034040 tx=0x7f022c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f0234ff9640 1 -- 172.21.15.7:0/1881264772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c036070 con 0x7f02381f4390 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f0234ff9640 1 -- 172.21.15.7:0/1881264772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022c037040 con 0x7f02381f4390 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f0234ff9640 1 -- 172.21.15.7:0/1881264772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c004030 con 0x7f02381f4390 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02380b4410 con 0x7f02381f4390 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.012+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02380b6850 con 0x7f02381f4390 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.014+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0238162e40 con 0x7f02381f4390 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.014+0000 7f0234ff9640 1 -- 172.21.15.7:0/1881264772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f022c0041d0 con 0x7f02381f4390 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.014+0000 7f0234ff9640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f022406c2e0 0x7f022406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.015+0000 7f0234ff9640 1 -- 172.21.15.7:0/1881264772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f022c0ab870 con 0x7f02381f4390 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.015+0000 7f0237fff640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f022406c2e0 0x7f022406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.015+0000 7f0237fff640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f022406c2e0 0x7f022406e7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02380c9df0 tx=0x7f0228009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:42.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.019+0000 7f0234ff9640 1 -- 172.21.15.7:0/1881264772 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f022c079770 con 0x7f02381f4390 2024-09-15T23:36:42.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.231+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f02380ba560 con 0x7f02381f4390 2024-09-15T23:36:42.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.878+0000 7f0234ff9640 1 -- 172.21.15.7:0/1881264772 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f022c07da70 con 0x7f02381f4390 2024-09-15T23:36:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.880+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f022406c2e0 msgr2=0x7f022406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.880+0000 7f023e511640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f022406c2e0 0x7f022406e7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02380c9df0 tx=0x7f0228009040 comp rx=0 tx=0).stop 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.880+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02381f4390 msgr2=0x7f02381f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.880+0000 7f023e511640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02381f4390 0x7f02381f4810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f022c034040 tx=0x7f022c004730 comp rx=0 tx=0).stop 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.881+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 shutdown_connections 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.881+0000 7f023e511640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02381f4d50 0x7f023815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.881+0000 7f023e511640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02381f4390 0x7f02381f4810 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.881+0000 7f023e511640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f022406c2e0 0x7f022406e7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.881+0000 7f023e511640 1 --2- 172.21.15.7:0/1881264772 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0220004800 0x7f0238161940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.881+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 >> 172.21.15.7:0/1881264772 conn(0x7f023808bfd0 msgr2=0x7f02380cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.881+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 shutdown_connections 2024-09-15T23:36:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:42.881+0000 7f023e511640 1 -- 172.21.15.7:0/1881264772 wait complete. 2024-09-15T23:36:42.896 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-15T23:36:42.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:36:43.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.003+0000 7f7526723640 1 Processor -- start 2024-09-15T23:36:43.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.003+0000 7f7526723640 1 -- start start 2024-09-15T23:36:43.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f7526723640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f752015e840 0x7f75200d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f7526723640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752006dd80 con 0x7f75200d0f90 2024-09-15T23:36:43.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f7526723640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75200cf560 con 0x7f75200d52e0 2024-09-15T23:36:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f7526723640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f75200cfe90 con 0x7f752015e840 2024-09-15T23:36:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f751ffff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f752015e840 0x7f75200d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f7524cc0640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f75200d0f90 0x7f75200d24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44614/0 (socket says 172.21.15.7:44614) 2024-09-15T23:36:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f7524cc0640 1 -- 172.21.15.7:0/3560656169 learned_addr learned my addr 172.21.15.7:0/3560656169 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f751ffff640 1 -- 172.21.15.7:0/3560656169 >> v1:172.21.15.18:6789/0 conn(0x7f75200d52e0 legacy=0x7f752006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f751ffff640 1 -- 172.21.15.7:0/3560656169 >> v1:172.21.15.7:6789/0 conn(0x7f75200d0f90 legacy=0x7f75200d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f751effd640 1 -- 172.21.15.7:0/3560656169 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3176417346 0 0) 0x7f752006dd80 con 0x7f75200d0f90 2024-09-15T23:36:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.004+0000 7f751ffff640 1 -- 172.21.15.7:0/3560656169 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75200cf840 con 0x7f752015e840 2024-09-15T23:36:43.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.005+0000 7f751ffff640 1 --2- 172.21.15.7:0/3560656169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752015e840 0x7f75200d0320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75100097a0 tx=0x7f751002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aba778b0514026e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:43.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.005+0000 7f751effd640 1 -- 172.21.15.7:0/3560656169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510037070 con 0x7f752015e840 2024-09-15T23:36:43.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.005+0000 7f751effd640 1 -- 172.21.15.7:0/3560656169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7510032440 con 0x7f752015e840 2024-09-15T23:36:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.005+0000 7f751effd640 1 -- 172.21.15.7:0/3560656169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510032810 con 0x7f752015e840 2024-09-15T23:36:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.006+0000 7f7526723640 1 -- 172.21.15.7:0/3560656169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752015e840 msgr2=0x7f75200d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.006+0000 7f7526723640 1 --2- 172.21.15.7:0/3560656169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752015e840 0x7f75200d0320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75100097a0 tx=0x7f751002a230 comp rx=0 tx=0).stop 2024-09-15T23:36:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.006+0000 7f7526723640 1 -- 172.21.15.7:0/3560656169 shutdown_connections 2024-09-15T23:36:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.006+0000 7f7526723640 1 --2- 172.21.15.7:0/3560656169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752015e840 0x7f75200d0320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.006+0000 7f7526723640 1 -- 172.21.15.7:0/3560656169 >> 172.21.15.7:0/3560656169 conn(0x7f752008bfd0 msgr2=0x7f75200d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.006+0000 7f7526723640 1 -- 172.21.15.7:0/3560656169 shutdown_connections 2024-09-15T23:36:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.006+0000 7f7526723640 1 -- 172.21.15.7:0/3560656169 wait complete. 2024-09-15T23:36:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.007+0000 7f7526723640 1 Processor -- start 2024-09-15T23:36:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.007+0000 7f7526723640 1 -- start start 2024-09-15T23:36:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.008+0000 7f7526723640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75200d0f90 0x7f752015a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.008+0000 7f7526723640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75200d52e0 0x7f75200ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.008+0000 7f7526723640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f752015e840 0x7f75200ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.008+0000 7f7526723640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f752015ec20 con 0x7f75200d52e0 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.008+0000 7f7526723640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f752006e030 con 0x7f75200d0f90 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.008+0000 7f7526723640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f75200c8d00 con 0x7f752015e840 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.008+0000 7f751f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75200d52e0 0x7f75200ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.008+0000 7f751f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75200d52e0 0x7f75200ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48052/0 (socket says 172.21.15.7:48052) 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.008+0000 7f751f7fe640 1 -- 172.21.15.7:0/3823715739 learned_addr learned my addr 172.21.15.7:0/3823715739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f7524cc0640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f752015e840 0x7f75200ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751ffff640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75200d0f90 0x7f752015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751f7fe640 1 -- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f752015e840 msgr2=0x7f75200ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751f7fe640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f752015e840 0x7f75200ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751f7fe640 1 -- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75200d0f90 msgr2=0x7f752015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751f7fe640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75200d0f90 0x7f752015a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751f7fe640 1 -- 172.21.15.7:0/3823715739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75200c80f0 con 0x7f75200d52e0 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751ffff640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75200d0f90 0x7f752015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f7524cc0640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f752015e840 0x7f75200ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751f7fe640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75200d52e0 0x7f75200ca310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f751400c450 tx=0x7f751400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751cff9640 1 -- 172.21.15.7:0/3823715739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75140052d0 con 0x7f75200d52e0 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751cff9640 1 -- 172.21.15.7:0/3823715739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7514005490 con 0x7f75200d52e0 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75200c83d0 con 0x7f75200d52e0 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f751cff9640 1 -- 172.21.15.7:0/3823715739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514003880 con 0x7f75200d52e0 2024-09-15T23:36:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.009+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75200c8990 con 0x7f75200d52e0 2024-09-15T23:36:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.010+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75200d08a0 con 0x7f75200d52e0 2024-09-15T23:36:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.011+0000 7f751cff9640 1 -- 172.21.15.7:0/3823715739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7514003a20 con 0x7f75200d52e0 2024-09-15T23:36:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.011+0000 7f751cff9640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f750c06c2e0 0x7f750c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.011+0000 7f751cff9640 1 -- 172.21.15.7:0/3823715739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f751408b410 con 0x7f75200d52e0 2024-09-15T23:36:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.011+0000 7f751ffff640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f750c06c2e0 0x7f750c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.012+0000 7f751ffff640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f750c06c2e0 0x7f750c06e7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7510034000 tx=0x7f751002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:43.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.015+0000 7f751cff9640 1 -- 172.21.15.7:0/3823715739 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7514059160 con 0x7f75200d52e0 2024-09-15T23:36:43.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.226+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75200d1cb0 con 0x7f75200d52e0 2024-09-15T23:36:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.226+0000 7f751cff9640 1 -- 172.21.15.7:0/3823715739 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5210 (secure 0 0 0) 0x7f751405d460 con 0x7f75200d52e0 2024-09-15T23:36:43.228 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:43.228 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-15T23:36:42:878666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:42.233279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:36:43.229 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-15T23:36:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f750c06c2e0 msgr2=0x7f750c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f750c06c2e0 0x7f750c06e7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7510034000 tx=0x7f751002e000 comp rx=0 tx=0).stop 2024-09-15T23:36:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75200d52e0 msgr2=0x7f75200ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75200d52e0 0x7f75200ca310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f751400c450 tx=0x7f751400c910 comp rx=0 tx=0).stop 2024-09-15T23:36:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 shutdown_connections 2024-09-15T23:36:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f750c06c2e0 0x7f750c06e7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f752015e840 0x7f75200ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75200d52e0 0x7f75200ca310 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 --2- 172.21.15.7:0/3823715739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75200d0f90 0x7f752015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 >> 172.21.15.7:0/3823715739 conn(0x7f752008bfd0 msgr2=0x7f75200b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 shutdown_connections 2024-09-15T23:36:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.228+0000 7f7526723640 1 -- 172.21.15.7:0/3823715739 wait complete. 2024-09-15T23:36:43.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.341+0000 7f8d050a5640 1 Processor -- start 2024-09-15T23:36:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.341+0000 7f8d050a5640 1 -- start start 2024-09-15T23:36:43.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.342+0000 7f8d050a5640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000d51c0 0x7f8d000d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.342+0000 7f8d050a5640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8d000d30d0 0x7f8d000b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.342+0000 7f8d050a5640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d000b3d10 con 0x7f8d0015e6d0 2024-09-15T23:36:43.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.342+0000 7f8d050a5640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8d000d28c0 con 0x7f8d000d30d0 2024-09-15T23:36:43.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.342+0000 7f8d050a5640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8d000d3540 con 0x7f8d000d51c0 2024-09-15T23:36:43.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.342+0000 7f8cfed9d640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8d0015e6d0 0x7f8d000d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44620/0 (socket says 172.21.15.7:44620) 2024-09-15T23:36:43.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.342+0000 7f8cfed9d640 1 -- 172.21.15.7:0/3629593553 learned_addr learned my addr 172.21.15.7:0/3629593553 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:43.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cff59e640 1 --2- 172.21.15.7:0/3629593553 >> v2:172.21.15.18:3300/0 conn(0x7f8d000d30d0 0x7f8d000b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfe59c640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000d51c0 0x7f8d000d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1203758336 0 0) 0x7f8d000b3d10 con 0x7f8d0015e6d0 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cec003520 con 0x7f8d0015e6d0 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 477060374 0 0) 0x7f8cec003520 con 0x7f8d0015e6d0 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000d51c0 msgr2=0x7f8d000d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000d51c0 0x7f8d000d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 >> v2:172.21.15.18:3300/0 conn(0x7f8d000d30d0 msgr2=0x7f8d000b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 --2- 172.21.15.7:0/3629593553 >> v2:172.21.15.18:3300/0 conn(0x7f8d000d30d0 0x7f8d000b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d000cfcd0 con 0x7f8d0015e6d0 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f8cf8001670 con 0x7f8d0015e6d0 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 >> v1:172.21.15.7:6789/0 conn(0x7f8d0015e6d0 legacy=0x7f8d000d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cec0039b0 0x7f8cec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec004800 0x7f8cec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000d51c0 0x7f8cec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d000d28c0 con 0x7f8cec004800 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8d000d3540 con 0x7f8cec0039b0 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8cec00b9c0 con 0x7f8d000d51c0 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfe59c640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec004800 0x7f8cec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfed9d640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cec0039b0 0x7f8cec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cff59e640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000d51c0 0x7f8cec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfe59c640 1 -- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000d51c0 msgr2=0x7f8cec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfe59c640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000d51c0 0x7f8cec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfe59c640 1 -- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cec0039b0 msgr2=0x7f8cec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfe59c640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cec0039b0 0x7f8cec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.343+0000 7f8cfe59c640 1 -- 172.21.15.7:0/3629593553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec003520 con 0x7f8cec004800 2024-09-15T23:36:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.344+0000 7f8cfe59c640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec004800 0x7f8cec004c80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009a90 tx=0x7f8cec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbca6eca1f5f4da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.344+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8036070 con 0x7f8cec004800 2024-09-15T23:36:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.344+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce8031400 con 0x7f8cec004800 2024-09-15T23:36:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.344+0000 7f8cfdd9b640 1 -- 172.21.15.7:0/3629593553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8009690 con 0x7f8cec004800 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.345+0000 7f8d050a5640 1 -- 172.21.15.7:0/3629593553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec004800 msgr2=0x7f8cec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.345+0000 7f8d050a5640 1 --2- 172.21.15.7:0/3629593553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec004800 0x7f8cec004c80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009a90 tx=0x7f8cec005830 comp rx=0 tx=0).stop 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.345+0000 7f8cff59e640 1 -- 172.21.15.7:0/3629593553 reap_dead start 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.345+0000 7f8d050a5640 1 -- 172.21.15.7:0/3629593553 shutdown_connections 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.345+0000 7f8d050a5640 1 -- 172.21.15.7:0/3629593553 >> 172.21.15.7:0/3629593553 conn(0x7f8d0008bfd0 msgr2=0x7f8d000d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.345+0000 7f8d050a5640 1 -- 172.21.15.7:0/3629593553 shutdown_connections 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.345+0000 7f8d050a5640 1 -- 172.21.15.7:0/3629593553 wait complete. 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.346+0000 7f8d050a5640 1 Processor -- start 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.346+0000 7f8d050a5640 1 -- start start 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8d050a5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cec004800 0x7f8d00159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8d050a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d000ce4a0 0x7f8d00158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8d050a5640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000cb6f0 0x7f8d000cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8d050a5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0015eab0 con 0x7f8d000ce4a0 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8d050a5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8d000d0230 con 0x7f8cec004800 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8d050a5640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8d000d0510 con 0x7f8d000cb6f0 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8cfe59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d000ce4a0 0x7f8d00158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8cfe59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d000ce4a0 0x7f8d00158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48070/0 (socket says 172.21.15.7:48070) 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8cfe59c640 1 -- 172.21.15.7:0/1637119465 learned_addr learned my addr 172.21.15.7:0/1637119465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8cff59e640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000cb6f0 0x7f8d000cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.347+0000 7f8cfed9d640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cec004800 0x7f8d00159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8cfe59c640 1 -- 172.21.15.7:0/1637119465 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000cb6f0 msgr2=0x7f8d000cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8cfe59c640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000cb6f0 0x7f8d000cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8cfe59c640 1 -- 172.21.15.7:0/1637119465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cec004800 msgr2=0x7f8d00159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8cfe59c640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cec004800 0x7f8d00159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8cfe59c640 1 -- 172.21.15.7:0/1637119465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d00159a70 con 0x7f8d000ce4a0 2024-09-15T23:36:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8cfe59c640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d000ce4a0 0x7f8d00158d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8034040 tx=0x7f8ce8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8cf77fe640 1 -- 172.21.15.7:0/1637119465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8041020 con 0x7f8d000ce4a0 2024-09-15T23:36:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8cf77fe640 1 -- 172.21.15.7:0/1637119465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce803c470 con 0x7f8d000ce4a0 2024-09-15T23:36:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8cf77fe640 1 -- 172.21.15.7:0/1637119465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8036040 con 0x7f8d000ce4a0 2024-09-15T23:36:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d00159d50 con 0x7f8d000ce4a0 2024-09-15T23:36:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.348+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0015a310 con 0x7f8d000ce4a0 2024-09-15T23:36:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.349+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d000d2c20 con 0x7f8d000ce4a0 2024-09-15T23:36:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.350+0000 7f8cf77fe640 1 -- 172.21.15.7:0/1637119465 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8ce8044020 con 0x7f8d000ce4a0 2024-09-15T23:36:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.350+0000 7f8cf77fe640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8ce006c2e0 0x7f8ce006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.350+0000 7f8cf77fe640 1 -- 172.21.15.7:0/1637119465 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8ce80ab740 con 0x7f8d000ce4a0 2024-09-15T23:36:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.350+0000 7f8cfed9d640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8ce006c2e0 0x7f8ce006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.351+0000 7f8cfed9d640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8ce006c2e0 0x7f8ce006e7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8007fd0 tx=0x7f8cf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.355+0000 7f8cf77fe640 1 -- 172.21.15.7:0/1637119465 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ce8079ae0 con 0x7f8d000ce4a0 2024-09-15T23:36:43.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.533+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d0006df80 con 0x7f8d000ce4a0 2024-09-15T23:36:43.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.534+0000 7f8cf77fe640 1 -- 172.21.15.7:0/1637119465 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7f8ce80793e0 con 0x7f8d000ce4a0 2024-09-15T23:36:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:43.536 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8ce006c2e0 msgr2=0x7f8ce006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8ce006c2e0 0x7f8ce006e7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8007fd0 tx=0x7f8cf8009040 comp rx=0 tx=0).stop 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d000ce4a0 msgr2=0x7f8d00158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d000ce4a0 0x7f8d00158d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8034040 tx=0x7f8ce8004730 comp rx=0 tx=0).stop 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 shutdown_connections 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8ce006c2e0 0x7f8ce006e7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8d000cb6f0 0x7f8d000cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d000ce4a0 0x7f8d00158d70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 --2- 172.21.15.7:0/1637119465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cec004800 0x7f8d00159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 >> 172.21.15.7:0/1637119465 conn(0x7f8d0008bfd0 msgr2=0x7f8d0006ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 shutdown_connections 2024-09-15T23:36:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.536+0000 7f8d050a5640 1 -- 172.21.15.7:0/1637119465 wait complete. 2024-09-15T23:36:43.550 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T23:36:43.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:36:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.653+0000 7f161808d640 1 Processor -- start 2024-09-15T23:36:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.653+0000 7f161808d640 1 -- start start 2024-09-15T23:36:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f161808d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1610161e60 0x7f16100d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f161808d640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f16100cfed0 0x7f16100d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f161808d640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16100d0300 con 0x7f16100d2b40 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f161808d640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f16100d5000 con 0x7f1610161e60 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f161808d640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f16100d1bb0 con 0x7f16100cfed0 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f1615e29640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1610161e60 0x7f16100d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f1615e29640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1610161e60 0x7f16100d51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48082/0 (socket says 172.21.15.7:48082) 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f161662a640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16100d2b40 0x7f16100b7420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44622/0 (socket says 172.21.15.7:44622) 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f1615e29640 1 -- 172.21.15.7:0/1453388317 learned_addr learned my addr 172.21.15.7:0/1453388317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f1615628640 1 --2- 172.21.15.7:0/1453388317 >> v2:172.21.15.18:3300/0 conn(0x7f16100cfed0 0x7f16100d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f1615e29640 1 -- 172.21.15.7:0/1453388317 >> v2:172.21.15.18:3300/0 conn(0x7f16100cfed0 msgr2=0x7f16100d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f1615e29640 1 --2- 172.21.15.7:0/1453388317 >> v2:172.21.15.18:3300/0 conn(0x7f16100cfed0 0x7f16100d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f1615e29640 1 -- 172.21.15.7:0/1453388317 >> v1:172.21.15.7:6789/0 conn(0x7f16100d2b40 legacy=0x7f16100b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.654+0000 7f1615e29640 1 -- 172.21.15.7:0/1453388317 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16100d05d0 con 0x7f1610161e60 2024-09-15T23:36:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.655+0000 7f1615e29640 1 --2- 172.21.15.7:0/1453388317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 0x7f16100d51a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f16100a2490 tx=0x7f160802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f1750adcb33cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.655+0000 7f1614e27640 1 -- 172.21.15.7:0/1453388317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608037070 con 0x7f1610161e60 2024-09-15T23:36:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.655+0000 7f1614e27640 1 -- 172.21.15.7:0/1453388317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160802ae50 con 0x7f1610161e60 2024-09-15T23:36:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.655+0000 7f1614e27640 1 -- 172.21.15.7:0/1453388317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608032860 con 0x7f1610161e60 2024-09-15T23:36:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.656+0000 7f161808d640 1 -- 172.21.15.7:0/1453388317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 msgr2=0x7f16100d51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.656+0000 7f161808d640 1 --2- 172.21.15.7:0/1453388317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 0x7f16100d51a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f16100a2490 tx=0x7f160802a300 comp rx=0 tx=0).stop 2024-09-15T23:36:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.656+0000 7f161808d640 1 -- 172.21.15.7:0/1453388317 shutdown_connections 2024-09-15T23:36:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.656+0000 7f161808d640 1 --2- 172.21.15.7:0/1453388317 >> v2:172.21.15.18:3300/0 conn(0x7f16100cfed0 0x7f16100d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.656+0000 7f161808d640 1 --2- 172.21.15.7:0/1453388317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 0x7f16100d51a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.656+0000 7f161808d640 1 -- 172.21.15.7:0/1453388317 >> 172.21.15.7:0/1453388317 conn(0x7f161008bfd0 msgr2=0x7f16100d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.656+0000 7f161808d640 1 -- 172.21.15.7:0/1453388317 shutdown_connections 2024-09-15T23:36:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.656+0000 7f161808d640 1 -- 172.21.15.7:0/1453388317 wait complete. 2024-09-15T23:36:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.657+0000 7f161808d640 1 Processor -- start 2024-09-15T23:36:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.657+0000 7f161808d640 1 -- start start 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161808d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100cfed0 0x7f1610156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161808d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f16100d2b40 0x7f1610159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161808d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 0x7f161015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161808d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16100d0e70 con 0x7f1610161e60 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161808d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f16100d0b90 con 0x7f16100cfed0 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161808d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f16100d1150 con 0x7f16100d2b40 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161662a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 0x7f161015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161662a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 0x7f161015a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48092/0 (socket says 172.21.15.7:48092) 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161662a640 1 -- 172.21.15.7:0/1449062798 learned_addr learned my addr 172.21.15.7:0/1449062798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f1615e29640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100cfed0 0x7f1610156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f1615628640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f16100d2b40 0x7f1610159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161662a640 1 -- 172.21.15.7:0/1449062798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f16100d2b40 msgr2=0x7f1610159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161662a640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f16100d2b40 0x7f1610159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161662a640 1 -- 172.21.15.7:0/1449062798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100cfed0 msgr2=0x7f1610156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161662a640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100cfed0 0x7f1610156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.658+0000 7f161662a640 1 -- 172.21.15.7:0/1449062798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1610156950 con 0x7f1610161e60 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.659+0000 7f161662a640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 0x7f161015a0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f160000c780 tx=0x7f160000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.659+0000 7f160e7fc640 1 -- 172.21.15.7:0/1449062798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600017070 con 0x7f1610161e60 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.659+0000 7f160e7fc640 1 -- 172.21.15.7:0/1449062798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160000a640 con 0x7f1610161e60 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.659+0000 7f160e7fc640 1 -- 172.21.15.7:0/1449062798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160000aa30 con 0x7f1610161e60 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.659+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161015e5a0 con 0x7f1610161e60 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.659+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1610158840 con 0x7f1610161e60 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.660+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161015fac0 con 0x7f1610161e60 2024-09-15T23:36:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.660+0000 7f160e7fc640 1 -- 172.21.15.7:0/1449062798 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f160001a070 con 0x7f1610161e60 2024-09-15T23:36:43.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.661+0000 7f160e7fc640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f15fc06c2b0 0x7f15fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:43.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.661+0000 7f160e7fc640 1 -- 172.21.15.7:0/1449062798 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f160008ac80 con 0x7f1610161e60 2024-09-15T23:36:43.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.661+0000 7f1615e29640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f15fc06c2b0 0x7f15fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:43.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.661+0000 7f1615e29640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f15fc06c2b0 0x7f15fc06e770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1608034000 tx=0x7f160802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.665+0000 7f160e7fc640 1 -- 172.21.15.7:0/1449062798 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f161015e5a0 con 0x7f1610161e60 2024-09-15T23:36:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.840+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16100d1670 con 0x7f15fc06c2b0 2024-09-15T23:36:43.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.848+0000 7f160e7fc640 1 -- 172.21.15.7:0/1449062798 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7f16100d1670 con 0x7f15fc06c2b0 2024-09-15T23:36:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:43.857 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:36:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f15fc06c2b0 msgr2=0x7f15fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f15fc06c2b0 0x7f15fc06e770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1608034000 tx=0x7f160802e000 comp rx=0 tx=0).stop 2024-09-15T23:36:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 msgr2=0x7f161015a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 0x7f161015a0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f160000c780 tx=0x7f160000cc40 comp rx=0 tx=0).stop 2024-09-15T23:36:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 shutdown_connections 2024-09-15T23:36:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1610161e60 0x7f161015a0c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f15fc06c2b0 0x7f15fc06e770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f16100d2b40 0x7f1610159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 --2- 172.21.15.7:0/1449062798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100cfed0 0x7f1610156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:43.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 >> 172.21.15.7:0/1449062798 conn(0x7f161008bfd0 msgr2=0x7f16100b7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:43.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 shutdown_connections 2024-09-15T23:36:43.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:43.852+0000 7f161808d640 1 -- 172.21.15.7:0/1449062798 wait complete. 2024-09-15T23:36:43.865 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-15T23:36:43.815857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252752,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721968,"statfs":{"total":773094113280,"available":772835295232,"internally_reserved":0,"allocated":3317760,"data_stored":2012508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002861"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623447+0000","last_change":"2024-09-15T23:36:22.558162+0000","last_active":"2024-09-15T23:36:28.623447+0000","last_peered":"2024-09-15T23:36:28.623447+0000","last_clean":"2024-09-15T23:36:28.623447+0000","last_became_active":"2024-09-15T23:36:22.557752+0000","last_became_peered":"2024-09-15T23:36:22.557752+0000","last_unstale":"2024-09-15T23:36:28.623447+0000","last_undegraded":"2024-09-15T23:36:28.623447+0000","last_fullsized":"2024-09-15T23:36:28.623447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:17.806236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024436+0000","last_change":"2024-09-15T23:36:23.591313+0000","last_active":"2024-09-15T23:36:30.024436+0000","last_peered":"2024-09-15T23:36:30.024436+0000","last_clean":"2024-09-15T23:36:30.024436+0000","last_became_active":"2024-09-15T23:36:23.590422+0000","last_became_peered":"2024-09-15T23:36:23.590422+0000","last_unstale":"2024-09-15T23:36:30.024436+0000","last_undegraded":"2024-09-15T23:36:30.024436+0000","last_fullsized":"2024-09-15T23:36:30.024436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454338+0000","last_change":"2024-09-15T23:36:27.609388+0000","last_active":"2024-09-15T23:36:29.454338+0000","last_peered":"2024-09-15T23:36:29.454338+0000","last_clean":"2024-09-15T23:36:29.454338+0000","last_became_active":"2024-09-15T23:36:27.608999+0000","last_became_peered":"2024-09-15T23:36:27.608999+0000","last_unstale":"2024-09-15T23:36:29.454338+0000","last_undegraded":"2024-09-15T23:36:29.454338+0000","last_fullsized":"2024-09-15T23:36:29.454338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:30.088730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024564+0000","last_change":"2024-09-15T23:36:22.562097+0000","last_active":"2024-09-15T23:36:30.024564+0000","last_peered":"2024-09-15T23:36:30.024564+0000","last_clean":"2024-09-15T23:36:30.024564+0000","last_became_active":"2024-09-15T23:36:22.561356+0000","last_became_peered":"2024-09-15T23:36:22.561356+0000","last_unstale":"2024-09-15T23:36:30.024564+0000","last_undegraded":"2024-09-15T23:36:30.024564+0000","last_fullsized":"2024-09-15T23:36:30.024564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:46.482072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024461+0000","last_change":"2024-09-15T23:36:23.589236+0000","last_active":"2024-09-15T23:36:30.024461+0000","last_peered":"2024-09-15T23:36:30.024461+0000","last_clean":"2024-09-15T23:36:30.024461+0000","last_became_active":"2024-09-15T23:36:23.588873+0000","last_became_peered":"2024-09-15T23:36:23.588873+0000","last_unstale":"2024-09-15T23:36:30.024461+0000","last_undegraded":"2024-09-15T23:36:30.024461+0000","last_fullsized":"2024-09-15T23:36:30.024461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:18.209937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024371+0000","last_change":"2024-09-15T23:36:27.611526+0000","last_active":"2024-09-15T23:36:30.024371+0000","last_peered":"2024-09-15T23:36:30.024371+0000","last_clean":"2024-09-15T23:36:30.024371+0000","last_became_active":"2024-09-15T23:36:27.610759+0000","last_became_peered":"2024-09-15T23:36:27.610759+0000","last_unstale":"2024-09-15T23:36:30.024371+0000","last_undegraded":"2024-09-15T23:36:30.024371+0000","last_fullsized":"2024-09-15T23:36:30.024371+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:06.783968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025217+0000","last_change":"2024-09-15T23:36:22.568198+0000","last_active":"2024-09-15T23:36:30.025217+0000","last_peered":"2024-09-15T23:36:30.025217+0000","last_clean":"2024-09-15T23:36:30.025217+0000","last_became_active":"2024-09-15T23:36:22.567888+0000","last_became_peered":"2024-09-15T23:36:22.567888+0000","last_unstale":"2024-09-15T23:36:30.025217+0000","last_undegraded":"2024-09-15T23:36:30.025217+0000","last_fullsized":"2024-09-15T23:36:30.025217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:14:47.610942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024413+0000","last_change":"2024-09-15T23:36:23.593279+0000","last_active":"2024-09-15T23:36:30.024413+0000","last_peered":"2024-09-15T23:36:30.024413+0000","last_clean":"2024-09-15T23:36:30.024413+0000","last_became_active":"2024-09-15T23:36:23.592921+0000","last_became_peered":"2024-09-15T23:36:23.592921+0000","last_unstale":"2024-09-15T23:36:30.024413+0000","last_undegraded":"2024-09-15T23:36:30.024413+0000","last_fullsized":"2024-09-15T23:36:30.024413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:40.194400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024320+0000","last_change":"2024-09-15T23:36:27.612903+0000","last_active":"2024-09-15T23:36:30.024320+0000","last_peered":"2024-09-15T23:36:30.024320+0000","last_clean":"2024-09-15T23:36:30.024320+0000","last_became_active":"2024-09-15T23:36:27.612441+0000","last_became_peered":"2024-09-15T23:36:27.612441+0000","last_unstale":"2024-09-15T23:36:30.024320+0000","last_undegraded":"2024-09-15T23:36:30.024320+0000","last_fullsized":"2024-09-15T23:36:30.024320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:43.647541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025323+0000","last_change":"2024-09-15T23:36:22.566488+0000","last_active":"2024-09-15T23:36:30.025323+0000","last_peered":"2024-09-15T23:36:30.025323+0000","last_clean":"2024-09-15T23:36:30.025323+0000","last_became_active":"2024-09-15T23:36:22.566016+0000","last_became_peered":"2024-09-15T23:36:22.566016+0000","last_unstale":"2024-09-15T23:36:30.025323+0000","last_undegraded":"2024-09-15T23:36:30.025323+0000","last_fullsized":"2024-09-15T23:36:30.025323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:45.276407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622706+0000","last_change":"2024-09-15T23:36:23.590254+0000","last_active":"2024-09-15T23:36:28.622706+0000","last_peered":"2024-09-15T23:36:28.622706+0000","last_clean":"2024-09-15T23:36:28.622706+0000","last_became_active":"2024-09-15T23:36:23.589902+0000","last_became_peered":"2024-09-15T23:36:23.589902+0000","last_unstale":"2024-09-15T23:36:28.622706+0000","last_undegraded":"2024-09-15T23:36:28.622706+0000","last_fullsized":"2024-09-15T23:36:28.622706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:37.196308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024145+0000","last_change":"2024-09-15T23:36:27.603610+0000","last_active":"2024-09-15T23:36:30.024145+0000","last_peered":"2024-09-15T23:36:30.024145+0000","last_clean":"2024-09-15T23:36:30.024145+0000","last_became_active":"2024-09-15T23:36:27.603177+0000","last_became_peered":"2024-09-15T23:36:27.603177+0000","last_unstale":"2024-09-15T23:36:30.024145+0000","last_undegraded":"2024-09-15T23:36:30.024145+0000","last_fullsized":"2024-09-15T23:36:30.024145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:55.070642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622285+0000","last_change":"2024-09-15T23:36:23.576248+0000","last_active":"2024-09-15T23:36:28.622285+0000","last_peered":"2024-09-15T23:36:28.622285+0000","last_clean":"2024-09-15T23:36:28.622285+0000","last_became_active":"2024-09-15T23:36:23.575715+0000","last_became_peered":"2024-09-15T23:36:23.575715+0000","last_unstale":"2024-09-15T23:36:28.622285+0000","last_undegraded":"2024-09-15T23:36:28.622285+0000","last_fullsized":"2024-09-15T23:36:28.622285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:59.081263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622550+0000","last_change":"2024-09-15T23:36:22.555862+0000","last_active":"2024-09-15T23:36:28.622550+0000","last_peered":"2024-09-15T23:36:28.622550+0000","last_clean":"2024-09-15T23:36:28.622550+0000","last_became_active":"2024-09-15T23:36:22.555589+0000","last_became_peered":"2024-09-15T23:36:22.555589+0000","last_unstale":"2024-09-15T23:36:28.622550+0000","last_undegraded":"2024-09-15T23:36:28.622550+0000","last_fullsized":"2024-09-15T23:36:28.622550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:44.229130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622578+0000","last_change":"2024-09-15T23:36:27.602404+0000","last_active":"2024-09-15T23:36:28.622578+0000","last_peered":"2024-09-15T23:36:28.622578+0000","last_clean":"2024-09-15T23:36:28.622578+0000","last_became_active":"2024-09-15T23:36:27.601991+0000","last_became_peered":"2024-09-15T23:36:27.601991+0000","last_unstale":"2024-09-15T23:36:28.622578+0000","last_undegraded":"2024-09-15T23:36:28.622578+0000","last_fullsized":"2024-09-15T23:36:28.622578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:00.662926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454749+0000","last_change":"2024-09-15T23:36:23.591252+0000","last_active":"2024-09-15T23:36:29.454749+0000","last_peered":"2024-09-15T23:36:29.454749+0000","last_clean":"2024-09-15T23:36:29.454749+0000","last_became_active":"2024-09-15T23:36:23.590976+0000","last_became_peered":"2024-09-15T23:36:23.590976+0000","last_unstale":"2024-09-15T23:36:29.454749+0000","last_undegraded":"2024-09-15T23:36:29.454749+0000","last_fullsized":"2024-09-15T23:36:29.454749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622161+0000","last_change":"2024-09-15T23:36:22.560352+0000","last_active":"2024-09-15T23:36:28.622161+0000","last_peered":"2024-09-15T23:36:28.622161+0000","last_clean":"2024-09-15T23:36:28.622161+0000","last_became_active":"2024-09-15T23:36:22.560052+0000","last_became_peered":"2024-09-15T23:36:22.560052+0000","last_unstale":"2024-09-15T23:36:28.622161+0000","last_undegraded":"2024-09-15T23:36:28.622161+0000","last_fullsized":"2024-09-15T23:36:28.622161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023452+0000","last_change":"2024-09-15T23:36:27.598358+0000","last_active":"2024-09-15T23:36:30.023452+0000","last_peered":"2024-09-15T23:36:30.023452+0000","last_clean":"2024-09-15T23:36:30.023452+0000","last_became_active":"2024-09-15T23:36:27.597941+0000","last_became_peered":"2024-09-15T23:36:27.597941+0000","last_unstale":"2024-09-15T23:36:30.023452+0000","last_undegraded":"2024-09-15T23:36:30.023452+0000","last_fullsized":"2024-09-15T23:36:30.023452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:54.115527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622271+0000","last_change":"2024-09-15T23:36:23.581091+0000","last_active":"2024-09-15T23:36:28.622271+0000","last_peered":"2024-09-15T23:36:28.622271+0000","last_clean":"2024-09-15T23:36:28.622271+0000","last_became_active":"2024-09-15T23:36:23.580807+0000","last_became_peered":"2024-09-15T23:36:23.580807+0000","last_unstale":"2024-09-15T23:36:28.622271+0000","last_undegraded":"2024-09-15T23:36:28.622271+0000","last_fullsized":"2024-09-15T23:36:28.622271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:01.465502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623733+0000","last_change":"2024-09-15T23:36:22.569473+0000","last_active":"2024-09-15T23:36:28.623733+0000","last_peered":"2024-09-15T23:36:28.623733+0000","last_clean":"2024-09-15T23:36:28.623733+0000","last_became_active":"2024-09-15T23:36:22.569168+0000","last_became_peered":"2024-09-15T23:36:22.569168+0000","last_unstale":"2024-09-15T23:36:28.623733+0000","last_undegraded":"2024-09-15T23:36:28.623733+0000","last_fullsized":"2024-09-15T23:36:28.623733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:46.065174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454933+0000","last_change":"2024-09-15T23:36:27.605801+0000","last_active":"2024-09-15T23:36:29.454933+0000","last_peered":"2024-09-15T23:36:29.454933+0000","last_clean":"2024-09-15T23:36:29.454933+0000","last_became_active":"2024-09-15T23:36:27.605416+0000","last_became_peered":"2024-09-15T23:36:27.605416+0000","last_unstale":"2024-09-15T23:36:29.454933+0000","last_undegraded":"2024-09-15T23:36:29.454933+0000","last_fullsized":"2024-09-15T23:36:29.454933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:27.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622056+0000","last_change":"2024-09-15T23:36:23.587550+0000","last_active":"2024-09-15T23:36:28.622056+0000","last_peered":"2024-09-15T23:36:28.622056+0000","last_clean":"2024-09-15T23:36:28.622056+0000","last_became_active":"2024-09-15T23:36:23.587213+0000","last_became_peered":"2024-09-15T23:36:23.587213+0000","last_unstale":"2024-09-15T23:36:28.622056+0000","last_undegraded":"2024-09-15T23:36:28.622056+0000","last_fullsized":"2024-09-15T23:36:28.622056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:15.353430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023245+0000","last_change":"2024-09-15T23:36:22.569502+0000","last_active":"2024-09-15T23:36:30.023245+0000","last_peered":"2024-09-15T23:36:30.023245+0000","last_clean":"2024-09-15T23:36:30.023245+0000","last_became_active":"2024-09-15T23:36:22.567715+0000","last_became_peered":"2024-09-15T23:36:22.567715+0000","last_unstale":"2024-09-15T23:36:30.023245+0000","last_undegraded":"2024-09-15T23:36:30.023245+0000","last_fullsized":"2024-09-15T23:36:30.023245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024741+0000","last_change":"2024-09-15T23:36:27.612217+0000","last_active":"2024-09-15T23:36:30.024741+0000","last_peered":"2024-09-15T23:36:30.024741+0000","last_clean":"2024-09-15T23:36:30.024741+0000","last_became_active":"2024-09-15T23:36:27.611783+0000","last_became_peered":"2024-09-15T23:36:27.611783+0000","last_unstale":"2024-09-15T23:36:30.024741+0000","last_undegraded":"2024-09-15T23:36:30.024741+0000","last_fullsized":"2024-09-15T23:36:30.024741+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:45.020968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623852+0000","last_change":"2024-09-15T23:36:23.576178+0000","last_active":"2024-09-15T23:36:28.623852+0000","last_peered":"2024-09-15T23:36:28.623852+0000","last_clean":"2024-09-15T23:36:28.623852+0000","last_became_active":"2024-09-15T23:36:23.575823+0000","last_became_peered":"2024-09-15T23:36:23.575823+0000","last_unstale":"2024-09-15T23:36:28.623852+0000","last_undegraded":"2024-09-15T23:36:28.623852+0000","last_fullsized":"2024-09-15T23:36:28.623852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:17.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623244+0000","last_change":"2024-09-15T23:36:22.570936+0000","last_active":"2024-09-15T23:36:28.623244+0000","last_peered":"2024-09-15T23:36:28.623244+0000","last_clean":"2024-09-15T23:36:28.623244+0000","last_became_active":"2024-09-15T23:36:22.569461+0000","last_became_peered":"2024-09-15T23:36:22.569461+0000","last_unstale":"2024-09-15T23:36:28.623244+0000","last_undegraded":"2024-09-15T23:36:28.623244+0000","last_fullsized":"2024-09-15T23:36:28.623244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:38.263694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024707+0000","last_change":"2024-09-15T23:36:27.605878+0000","last_active":"2024-09-15T23:36:30.024707+0000","last_peered":"2024-09-15T23:36:30.024707+0000","last_clean":"2024-09-15T23:36:30.024707+0000","last_became_active":"2024-09-15T23:36:27.605448+0000","last_became_peered":"2024-09-15T23:36:27.605448+0000","last_unstale":"2024-09-15T23:36:30.024707+0000","last_undegraded":"2024-09-15T23:36:30.024707+0000","last_fullsized":"2024-09-15T23:36:30.024707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:00.293850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623345+0000","last_change":"2024-09-15T23:36:23.590802+0000","last_active":"2024-09-15T23:36:28.623345+0000","last_peered":"2024-09-15T23:36:28.623345+0000","last_clean":"2024-09-15T23:36:28.623345+0000","last_became_active":"2024-09-15T23:36:23.590505+0000","last_became_peered":"2024-09-15T23:36:23.590505+0000","last_unstale":"2024-09-15T23:36:28.623345+0000","last_undegraded":"2024-09-15T23:36:28.623345+0000","last_fullsized":"2024-09-15T23:36:28.623345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:50.950584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624097+0000","last_change":"2024-09-15T23:36:22.569675+0000","last_active":"2024-09-15T23:36:28.624097+0000","last_peered":"2024-09-15T23:36:28.624097+0000","last_clean":"2024-09-15T23:36:28.624097+0000","last_became_active":"2024-09-15T23:36:22.569386+0000","last_became_peered":"2024-09-15T23:36:22.569386+0000","last_unstale":"2024-09-15T23:36:28.624097+0000","last_undegraded":"2024-09-15T23:36:28.624097+0000","last_fullsized":"2024-09-15T23:36:28.624097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:00.420124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622250+0000","last_change":"2024-09-15T23:36:27.601084+0000","last_active":"2024-09-15T23:36:28.622250+0000","last_peered":"2024-09-15T23:36:28.622250+0000","last_clean":"2024-09-15T23:36:28.622250+0000","last_became_active":"2024-09-15T23:36:27.600671+0000","last_became_peered":"2024-09-15T23:36:27.600671+0000","last_unstale":"2024-09-15T23:36:28.622250+0000","last_undegraded":"2024-09-15T23:36:28.622250+0000","last_fullsized":"2024-09-15T23:36:28.622250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:28.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622694+0000","last_change":"2024-09-15T23:36:23.595285+0000","last_active":"2024-09-15T23:36:28.622694+0000","last_peered":"2024-09-15T23:36:28.622694+0000","last_clean":"2024-09-15T23:36:28.622694+0000","last_became_active":"2024-09-15T23:36:23.594596+0000","last_became_peered":"2024-09-15T23:36:23.594596+0000","last_unstale":"2024-09-15T23:36:28.622694+0000","last_undegraded":"2024-09-15T23:36:28.622694+0000","last_fullsized":"2024-09-15T23:36:28.622694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:39.265619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623245+0000","last_change":"2024-09-15T23:36:22.553052+0000","last_active":"2024-09-15T23:36:28.623245+0000","last_peered":"2024-09-15T23:36:28.623245+0000","last_clean":"2024-09-15T23:36:28.623245+0000","last_became_active":"2024-09-15T23:36:22.552766+0000","last_became_peered":"2024-09-15T23:36:22.552766+0000","last_unstale":"2024-09-15T23:36:28.623245+0000","last_undegraded":"2024-09-15T23:36:28.623245+0000","last_fullsized":"2024-09-15T23:36:28.623245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:45.928194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621909+0000","last_change":"2024-09-15T23:36:27.601501+0000","last_active":"2024-09-15T23:36:28.621909+0000","last_peered":"2024-09-15T23:36:28.621909+0000","last_clean":"2024-09-15T23:36:28.621909+0000","last_became_active":"2024-09-15T23:36:27.601071+0000","last_became_peered":"2024-09-15T23:36:27.601071+0000","last_unstale":"2024-09-15T23:36:28.621909+0000","last_undegraded":"2024-09-15T23:36:28.621909+0000","last_fullsized":"2024-09-15T23:36:28.621909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:03.241943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024193+0000","last_change":"2024-09-15T23:36:23.591415+0000","last_active":"2024-09-15T23:36:30.024193+0000","last_peered":"2024-09-15T23:36:30.024193+0000","last_clean":"2024-09-15T23:36:30.024193+0000","last_became_active":"2024-09-15T23:36:23.591132+0000","last_became_peered":"2024-09-15T23:36:23.591132+0000","last_unstale":"2024-09-15T23:36:30.024193+0000","last_undegraded":"2024-09-15T23:36:30.024193+0000","last_fullsized":"2024-09-15T23:36:30.024193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024034+0000","last_change":"2024-09-15T23:36:22.561640+0000","last_active":"2024-09-15T23:36:30.024034+0000","last_peered":"2024-09-15T23:36:30.024034+0000","last_clean":"2024-09-15T23:36:30.024034+0000","last_became_active":"2024-09-15T23:36:22.561218+0000","last_became_peered":"2024-09-15T23:36:22.561218+0000","last_unstale":"2024-09-15T23:36:30.024034+0000","last_undegraded":"2024-09-15T23:36:30.024034+0000","last_fullsized":"2024-09-15T23:36:30.024034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:06:31.226510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621863+0000","last_change":"2024-09-15T23:36:27.606889+0000","last_active":"2024-09-15T23:36:28.621863+0000","last_peered":"2024-09-15T23:36:28.621863+0000","last_clean":"2024-09-15T23:36:28.621863+0000","last_became_active":"2024-09-15T23:36:27.606466+0000","last_became_peered":"2024-09-15T23:36:27.606466+0000","last_unstale":"2024-09-15T23:36:28.621863+0000","last_undegraded":"2024-09-15T23:36:28.621863+0000","last_fullsized":"2024-09-15T23:36:28.621863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:55.070642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023954+0000","last_change":"2024-09-15T23:36:23.596605+0000","last_active":"2024-09-15T23:36:30.023954+0000","last_peered":"2024-09-15T23:36:30.023954+0000","last_clean":"2024-09-15T23:36:30.023954+0000","last_became_active":"2024-09-15T23:36:23.596318+0000","last_became_peered":"2024-09-15T23:36:23.596318+0000","last_unstale":"2024-09-15T23:36:30.023954+0000","last_undegraded":"2024-09-15T23:36:30.023954+0000","last_fullsized":"2024-09-15T23:36:30.023954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:02.746761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622853+0000","last_change":"2024-09-15T23:36:22.559703+0000","last_active":"2024-09-15T23:36:28.622853+0000","last_peered":"2024-09-15T23:36:28.622853+0000","last_clean":"2024-09-15T23:36:28.622853+0000","last_became_active":"2024-09-15T23:36:22.559271+0000","last_became_peered":"2024-09-15T23:36:22.559271+0000","last_unstale":"2024-09-15T23:36:28.622853+0000","last_undegraded":"2024-09-15T23:36:28.622853+0000","last_fullsized":"2024-09-15T23:36:28.622853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:23.347088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622571+0000","last_change":"2024-09-15T23:36:27.598188+0000","last_active":"2024-09-15T23:36:28.622571+0000","last_peered":"2024-09-15T23:36:28.622571+0000","last_clean":"2024-09-15T23:36:28.622571+0000","last_became_active":"2024-09-15T23:36:27.596219+0000","last_became_peered":"2024-09-15T23:36:27.596219+0000","last_unstale":"2024-09-15T23:36:28.622571+0000","last_undegraded":"2024-09-15T23:36:28.622571+0000","last_fullsized":"2024-09-15T23:36:28.622571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:23.776974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023619+0000","last_change":"2024-09-15T23:36:23.582672+0000","last_active":"2024-09-15T23:36:30.023619+0000","last_peered":"2024-09-15T23:36:30.023619+0000","last_clean":"2024-09-15T23:36:30.023619+0000","last_became_active":"2024-09-15T23:36:23.582388+0000","last_became_peered":"2024-09-15T23:36:23.582388+0000","last_unstale":"2024-09-15T23:36:30.023619+0000","last_undegraded":"2024-09-15T23:36:30.023619+0000","last_fullsized":"2024-09-15T23:36:30.023619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:07.005224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454431+0000","last_change":"2024-09-15T23:36:22.561458+0000","last_active":"2024-09-15T23:36:29.454431+0000","last_peered":"2024-09-15T23:36:29.454431+0000","last_clean":"2024-09-15T23:36:29.454431+0000","last_became_active":"2024-09-15T23:36:22.561171+0000","last_became_peered":"2024-09-15T23:36:22.561171+0000","last_unstale":"2024-09-15T23:36:29.454431+0000","last_undegraded":"2024-09-15T23:36:29.454431+0000","last_fullsized":"2024-09-15T23:36:29.454431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622564+0000","last_change":"2024-09-15T23:36:27.597617+0000","last_active":"2024-09-15T23:36:28.622564+0000","last_peered":"2024-09-15T23:36:28.622564+0000","last_clean":"2024-09-15T23:36:28.622564+0000","last_became_active":"2024-09-15T23:36:27.597224+0000","last_became_peered":"2024-09-15T23:36:27.597224+0000","last_unstale":"2024-09-15T23:36:28.622564+0000","last_undegraded":"2024-09-15T23:36:28.622564+0000","last_fullsized":"2024-09-15T23:36:28.622564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:06.783968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621714+0000","last_change":"2024-09-15T23:36:23.590567+0000","last_active":"2024-09-15T23:36:28.621714+0000","last_peered":"2024-09-15T23:36:28.621714+0000","last_clean":"2024-09-15T23:36:28.621714+0000","last_became_active":"2024-09-15T23:36:23.590272+0000","last_became_peered":"2024-09-15T23:36:23.590272+0000","last_unstale":"2024-09-15T23:36:28.621714+0000","last_undegraded":"2024-09-15T23:36:28.621714+0000","last_fullsized":"2024-09-15T23:36:28.621714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622169+0000","last_change":"2024-09-15T23:36:22.560601+0000","last_active":"2024-09-15T23:36:28.622169+0000","last_peered":"2024-09-15T23:36:28.622169+0000","last_clean":"2024-09-15T23:36:28.622169+0000","last_became_active":"2024-09-15T23:36:22.560306+0000","last_became_peered":"2024-09-15T23:36:22.560306+0000","last_unstale":"2024-09-15T23:36:28.622169+0000","last_undegraded":"2024-09-15T23:36:28.622169+0000","last_fullsized":"2024-09-15T23:36:28.622169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:38.283320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024190+0000","last_change":"2024-09-15T23:36:27.610042+0000","last_active":"2024-09-15T23:36:30.024190+0000","last_peered":"2024-09-15T23:36:30.024190+0000","last_clean":"2024-09-15T23:36:30.024190+0000","last_became_active":"2024-09-15T23:36:27.609630+0000","last_became_peered":"2024-09-15T23:36:27.609630+0000","last_unstale":"2024-09-15T23:36:30.024190+0000","last_undegraded":"2024-09-15T23:36:30.024190+0000","last_fullsized":"2024-09-15T23:36:30.024190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:03.827160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624171+0000","last_change":"2024-09-15T23:36:23.580635+0000","last_active":"2024-09-15T23:36:28.624171+0000","last_peered":"2024-09-15T23:36:28.624171+0000","last_clean":"2024-09-15T23:36:28.624171+0000","last_became_active":"2024-09-15T23:36:23.580348+0000","last_became_peered":"2024-09-15T23:36:23.580348+0000","last_unstale":"2024-09-15T23:36:28.624171+0000","last_undegraded":"2024-09-15T23:36:28.624171+0000","last_fullsized":"2024-09-15T23:36:28.624171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:03.646324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622420+0000","last_change":"2024-09-15T23:36:22.552162+0000","last_active":"2024-09-15T23:36:28.622420+0000","last_peered":"2024-09-15T23:36:28.622420+0000","last_clean":"2024-09-15T23:36:28.622420+0000","last_became_active":"2024-09-15T23:36:22.551788+0000","last_became_peered":"2024-09-15T23:36:22.551788+0000","last_unstale":"2024-09-15T23:36:28.622420+0000","last_undegraded":"2024-09-15T23:36:28.622420+0000","last_fullsized":"2024-09-15T23:36:28.622420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:04.332504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023778+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:36:30.023778+0000","last_peered":"2024-09-15T23:36:30.023778+0000","last_clean":"2024-09-15T23:36:30.023778+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:36:30.023778+0000","last_undegraded":"2024-09-15T23:36:30.023778+0000","last_fullsized":"2024-09-15T23:36:30.023778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623010+0000","last_change":"2024-09-15T23:36:27.604371+0000","last_active":"2024-09-15T23:36:28.623010+0000","last_peered":"2024-09-15T23:36:28.623010+0000","last_clean":"2024-09-15T23:36:28.623010+0000","last_became_active":"2024-09-15T23:36:27.603926+0000","last_became_peered":"2024-09-15T23:36:27.603926+0000","last_unstale":"2024-09-15T23:36:28.623010+0000","last_undegraded":"2024-09-15T23:36:28.623010+0000","last_fullsized":"2024-09-15T23:36:28.623010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:41.910981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622897+0000","last_change":"2024-09-15T23:36:23.575297+0000","last_active":"2024-09-15T23:36:28.622897+0000","last_peered":"2024-09-15T23:36:28.622897+0000","last_clean":"2024-09-15T23:36:28.622897+0000","last_became_active":"2024-09-15T23:36:23.574997+0000","last_became_peered":"2024-09-15T23:36:23.574997+0000","last_unstale":"2024-09-15T23:36:28.622897+0000","last_undegraded":"2024-09-15T23:36:28.622897+0000","last_fullsized":"2024-09-15T23:36:28.622897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:24.495389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622829+0000","last_change":"2024-09-15T23:36:22.569366+0000","last_active":"2024-09-15T23:36:28.622829+0000","last_peered":"2024-09-15T23:36:28.622829+0000","last_clean":"2024-09-15T23:36:28.622829+0000","last_became_active":"2024-09-15T23:36:22.569043+0000","last_became_peered":"2024-09-15T23:36:22.569043+0000","last_unstale":"2024-09-15T23:36:28.622829+0000","last_undegraded":"2024-09-15T23:36:28.622829+0000","last_fullsized":"2024-09-15T23:36:28.622829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:55.907462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024315+0000","last_change":"2024-09-15T23:36:27.611141+0000","last_active":"2024-09-15T23:36:30.024315+0000","last_peered":"2024-09-15T23:36:30.024315+0000","last_clean":"2024-09-15T23:36:30.024315+0000","last_became_active":"2024-09-15T23:36:27.610654+0000","last_became_peered":"2024-09-15T23:36:27.610654+0000","last_unstale":"2024-09-15T23:36:30.024315+0000","last_undegraded":"2024-09-15T23:36:30.024315+0000","last_fullsized":"2024-09-15T23:36:30.024315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:03.241943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025277+0000","last_change":"2024-09-15T23:36:23.592044+0000","last_active":"2024-09-15T23:36:30.025277+0000","last_peered":"2024-09-15T23:36:30.025277+0000","last_clean":"2024-09-15T23:36:30.025277+0000","last_became_active":"2024-09-15T23:36:23.591419+0000","last_became_peered":"2024-09-15T23:36:23.591419+0000","last_unstale":"2024-09-15T23:36:30.025277+0000","last_undegraded":"2024-09-15T23:36:30.025277+0000","last_fullsized":"2024-09-15T23:36:30.025277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:59.204736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024210+0000","last_change":"2024-09-15T23:36:22.568129+0000","last_active":"2024-09-15T23:36:30.024210+0000","last_peered":"2024-09-15T23:36:30.024210+0000","last_clean":"2024-09-15T23:36:30.024210+0000","last_became_active":"2024-09-15T23:36:22.567909+0000","last_became_peered":"2024-09-15T23:36:22.567909+0000","last_unstale":"2024-09-15T23:36:30.024210+0000","last_undegraded":"2024-09-15T23:36:30.024210+0000","last_fullsized":"2024-09-15T23:36:30.024210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:46.065174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024320+0000","last_change":"2024-09-15T23:36:27.610796+0000","last_active":"2024-09-15T23:36:30.024320+0000","last_peered":"2024-09-15T23:36:30.024320+0000","last_clean":"2024-09-15T23:36:30.024320+0000","last_became_active":"2024-09-15T23:36:27.610355+0000","last_became_peered":"2024-09-15T23:36:27.610355+0000","last_unstale":"2024-09-15T23:36:30.024320+0000","last_undegraded":"2024-09-15T23:36:30.024320+0000","last_fullsized":"2024-09-15T23:36:30.024320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:22.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024449+0000","last_change":"2024-09-15T23:36:23.592411+0000","last_active":"2024-09-15T23:36:30.024449+0000","last_peered":"2024-09-15T23:36:30.024449+0000","last_clean":"2024-09-15T23:36:30.024449+0000","last_became_active":"2024-09-15T23:36:23.592130+0000","last_became_peered":"2024-09-15T23:36:23.592130+0000","last_unstale":"2024-09-15T23:36:30.024449+0000","last_undegraded":"2024-09-15T23:36:30.024449+0000","last_fullsized":"2024-09-15T23:36:30.024449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.872038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623974+0000","last_change":"2024-09-15T23:36:22.560112+0000","last_active":"2024-09-15T23:36:28.623974+0000","last_peered":"2024-09-15T23:36:28.623974+0000","last_clean":"2024-09-15T23:36:28.623974+0000","last_became_active":"2024-09-15T23:36:22.559811+0000","last_became_peered":"2024-09-15T23:36:22.559811+0000","last_unstale":"2024-09-15T23:36:28.623974+0000","last_undegraded":"2024-09-15T23:36:28.623974+0000","last_fullsized":"2024-09-15T23:36:28.623974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:46.482072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621872+0000","last_change":"2024-09-15T23:36:27.603263+0000","last_active":"2024-09-15T23:36:28.621872+0000","last_peered":"2024-09-15T23:36:28.621872+0000","last_clean":"2024-09-15T23:36:28.621872+0000","last_became_active":"2024-09-15T23:36:27.602856+0000","last_became_peered":"2024-09-15T23:36:27.602856+0000","last_unstale":"2024-09-15T23:36:28.621872+0000","last_undegraded":"2024-09-15T23:36:28.621872+0000","last_fullsized":"2024-09-15T23:36:28.621872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:38.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024861+0000","last_change":"2024-09-15T23:36:23.594876+0000","last_active":"2024-09-15T23:36:30.024861+0000","last_peered":"2024-09-15T23:36:30.024861+0000","last_clean":"2024-09-15T23:36:30.024861+0000","last_became_active":"2024-09-15T23:36:23.594592+0000","last_became_peered":"2024-09-15T23:36:23.594592+0000","last_unstale":"2024-09-15T23:36:30.024861+0000","last_undegraded":"2024-09-15T23:36:30.024861+0000","last_fullsized":"2024-09-15T23:36:30.024861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:05.334429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024659+0000","last_change":"2024-09-15T23:36:22.568643+0000","last_active":"2024-09-15T23:36:30.024659+0000","last_peered":"2024-09-15T23:36:30.024659+0000","last_clean":"2024-09-15T23:36:30.024659+0000","last_became_active":"2024-09-15T23:36:22.568351+0000","last_became_peered":"2024-09-15T23:36:22.568351+0000","last_unstale":"2024-09-15T23:36:30.024659+0000","last_undegraded":"2024-09-15T23:36:30.024659+0000","last_fullsized":"2024-09-15T23:36:30.024659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:39.192475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023472+0000","last_change":"2024-09-15T23:36:27.605261+0000","last_active":"2024-09-15T23:36:30.023472+0000","last_peered":"2024-09-15T23:36:30.023472+0000","last_clean":"2024-09-15T23:36:30.023472+0000","last_became_active":"2024-09-15T23:36:27.604780+0000","last_became_peered":"2024-09-15T23:36:27.604780+0000","last_unstale":"2024-09-15T23:36:30.023472+0000","last_undegraded":"2024-09-15T23:36:30.023472+0000","last_fullsized":"2024-09-15T23:36:30.023472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:09.371636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621977+0000","last_change":"2024-09-15T23:36:23.582678+0000","last_active":"2024-09-15T23:36:28.621977+0000","last_peered":"2024-09-15T23:36:28.621977+0000","last_clean":"2024-09-15T23:36:28.621977+0000","last_became_active":"2024-09-15T23:36:23.582022+0000","last_became_peered":"2024-09-15T23:36:23.582022+0000","last_unstale":"2024-09-15T23:36:28.621977+0000","last_undegraded":"2024-09-15T23:36:28.621977+0000","last_fullsized":"2024-09-15T23:36:28.621977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:14.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622315+0000","last_change":"2024-09-15T23:36:22.552625+0000","last_active":"2024-09-15T23:36:28.622315+0000","last_peered":"2024-09-15T23:36:28.622315+0000","last_clean":"2024-09-15T23:36:28.622315+0000","last_became_active":"2024-09-15T23:36:22.552330+0000","last_became_peered":"2024-09-15T23:36:22.552330+0000","last_unstale":"2024-09-15T23:36:28.622315+0000","last_undegraded":"2024-09-15T23:36:28.622315+0000","last_fullsized":"2024-09-15T23:36:28.622315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:06:31.226510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024467+0000","last_change":"2024-09-15T23:36:27.607752+0000","last_active":"2024-09-15T23:36:30.024467+0000","last_peered":"2024-09-15T23:36:30.024467+0000","last_clean":"2024-09-15T23:36:30.024467+0000","last_became_active":"2024-09-15T23:36:27.607269+0000","last_became_peered":"2024-09-15T23:36:27.607269+0000","last_unstale":"2024-09-15T23:36:30.024467+0000","last_undegraded":"2024-09-15T23:36:30.024467+0000","last_fullsized":"2024-09-15T23:36:30.024467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:55.005028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023687+0000","last_change":"2024-09-15T23:36:23.585722+0000","last_active":"2024-09-15T23:36:30.023687+0000","last_peered":"2024-09-15T23:36:30.023687+0000","last_clean":"2024-09-15T23:36:30.023687+0000","last_became_active":"2024-09-15T23:36:23.585435+0000","last_became_peered":"2024-09-15T23:36:23.585435+0000","last_unstale":"2024-09-15T23:36:30.023687+0000","last_undegraded":"2024-09-15T23:36:30.023687+0000","last_fullsized":"2024-09-15T23:36:30.023687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023646+0000","last_change":"2024-09-15T23:36:22.568106+0000","last_active":"2024-09-15T23:36:30.023646+0000","last_peered":"2024-09-15T23:36:30.023646+0000","last_clean":"2024-09-15T23:36:30.023646+0000","last_became_active":"2024-09-15T23:36:22.567770+0000","last_became_peered":"2024-09-15T23:36:22.567770+0000","last_unstale":"2024-09-15T23:36:30.023646+0000","last_undegraded":"2024-09-15T23:36:30.023646+0000","last_fullsized":"2024-09-15T23:36:30.023646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:57.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024431+0000","last_change":"2024-09-15T23:36:27.611234+0000","last_active":"2024-09-15T23:36:30.024431+0000","last_peered":"2024-09-15T23:36:30.024431+0000","last_clean":"2024-09-15T23:36:30.024431+0000","last_became_active":"2024-09-15T23:36:27.610828+0000","last_became_peered":"2024-09-15T23:36:27.610828+0000","last_unstale":"2024-09-15T23:36:30.024431+0000","last_undegraded":"2024-09-15T23:36:30.024431+0000","last_fullsized":"2024-09-15T23:36:30.024431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:30.372516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622286+0000","last_change":"2024-09-15T23:36:23.592816+0000","last_active":"2024-09-15T23:36:28.622286+0000","last_peered":"2024-09-15T23:36:28.622286+0000","last_clean":"2024-09-15T23:36:28.622286+0000","last_became_active":"2024-09-15T23:36:23.590219+0000","last_became_peered":"2024-09-15T23:36:23.590219+0000","last_unstale":"2024-09-15T23:36:28.622286+0000","last_undegraded":"2024-09-15T23:36:28.622286+0000","last_fullsized":"2024-09-15T23:36:28.622286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:09:38.065436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622572+0000","last_change":"2024-09-15T23:36:22.555612+0000","last_active":"2024-09-15T23:36:28.622572+0000","last_peered":"2024-09-15T23:36:28.622572+0000","last_clean":"2024-09-15T23:36:28.622572+0000","last_became_active":"2024-09-15T23:36:22.555279+0000","last_became_peered":"2024-09-15T23:36:22.555279+0000","last_unstale":"2024-09-15T23:36:28.622572+0000","last_undegraded":"2024-09-15T23:36:28.622572+0000","last_fullsized":"2024-09-15T23:36:28.622572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:18.004944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622407+0000","last_change":"2024-09-15T23:36:27.594766+0000","last_active":"2024-09-15T23:36:28.622407+0000","last_peered":"2024-09-15T23:36:28.622407+0000","last_clean":"2024-09-15T23:36:28.622407+0000","last_became_active":"2024-09-15T23:36:27.594179+0000","last_became_peered":"2024-09-15T23:36:27.594179+0000","last_unstale":"2024-09-15T23:36:28.622407+0000","last_undegraded":"2024-09-15T23:36:28.622407+0000","last_fullsized":"2024-09-15T23:36:28.622407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:43.647541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622052+0000","last_change":"2024-09-15T23:36:23.595059+0000","last_active":"2024-09-15T23:36:28.622052+0000","last_peered":"2024-09-15T23:36:28.622052+0000","last_clean":"2024-09-15T23:36:28.622052+0000","last_became_active":"2024-09-15T23:36:23.594760+0000","last_became_peered":"2024-09-15T23:36:23.594760+0000","last_unstale":"2024-09-15T23:36:28.622052+0000","last_undegraded":"2024-09-15T23:36:28.622052+0000","last_fullsized":"2024-09-15T23:36:28.622052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:20.169043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454742+0000","last_change":"2024-09-15T23:36:22.569556+0000","last_active":"2024-09-15T23:36:29.454742+0000","last_peered":"2024-09-15T23:36:29.454742+0000","last_clean":"2024-09-15T23:36:29.454742+0000","last_became_active":"2024-09-15T23:36:22.569214+0000","last_became_peered":"2024-09-15T23:36:22.569214+0000","last_unstale":"2024-09-15T23:36:29.454742+0000","last_undegraded":"2024-09-15T23:36:29.454742+0000","last_fullsized":"2024-09-15T23:36:29.454742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:14:47.610942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622749+0000","last_change":"2024-09-15T23:36:27.601370+0000","last_active":"2024-09-15T23:36:28.622749+0000","last_peered":"2024-09-15T23:36:28.622749+0000","last_clean":"2024-09-15T23:36:28.622749+0000","last_became_active":"2024-09-15T23:36:27.600995+0000","last_became_peered":"2024-09-15T23:36:27.600995+0000","last_unstale":"2024-09-15T23:36:28.622749+0000","last_undegraded":"2024-09-15T23:36:28.622749+0000","last_fullsized":"2024-09-15T23:36:28.622749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:13.579126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025128+0000","last_change":"2024-09-15T23:36:23.595877+0000","last_active":"2024-09-15T23:36:30.025128+0000","last_peered":"2024-09-15T23:36:30.025128+0000","last_clean":"2024-09-15T23:36:30.025128+0000","last_became_active":"2024-09-15T23:36:23.595086+0000","last_became_peered":"2024-09-15T23:36:23.595086+0000","last_unstale":"2024-09-15T23:36:30.025128+0000","last_undegraded":"2024-09-15T23:36:30.025128+0000","last_fullsized":"2024-09-15T23:36:30.025128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:19.739767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:33.124757+0000","last_change":"2024-09-15T23:36:22.551636+0000","last_active":"2024-09-15T23:36:33.124757+0000","last_peered":"2024-09-15T23:36:33.124757+0000","last_clean":"2024-09-15T23:36:33.124757+0000","last_became_active":"2024-09-15T23:36:22.551316+0000","last_became_peered":"2024-09-15T23:36:22.551316+0000","last_unstale":"2024-09-15T23:36:33.124757+0000","last_undegraded":"2024-09-15T23:36:33.124757+0000","last_fullsized":"2024-09-15T23:36:33.124757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624158+0000","last_change":"2024-09-15T23:36:27.606629+0000","last_active":"2024-09-15T23:36:28.624158+0000","last_peered":"2024-09-15T23:36:28.624158+0000","last_clean":"2024-09-15T23:36:28.624158+0000","last_became_active":"2024-09-15T23:36:27.606224+0000","last_became_peered":"2024-09-15T23:36:27.606224+0000","last_unstale":"2024-09-15T23:36:28.624158+0000","last_undegraded":"2024-09-15T23:36:28.624158+0000","last_fullsized":"2024-09-15T23:36:28.624158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:09.576731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622435+0000","last_change":"2024-09-15T23:36:23.580904+0000","last_active":"2024-09-15T23:36:28.622435+0000","last_peered":"2024-09-15T23:36:28.622435+0000","last_clean":"2024-09-15T23:36:28.622435+0000","last_became_active":"2024-09-15T23:36:23.580459+0000","last_became_peered":"2024-09-15T23:36:23.580459+0000","last_unstale":"2024-09-15T23:36:28.622435+0000","last_undegraded":"2024-09-15T23:36:28.622435+0000","last_fullsized":"2024-09-15T23:36:28.622435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:50.078320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454752+0000","last_change":"2024-09-15T23:36:22.557264+0000","last_active":"2024-09-15T23:36:29.454752+0000","last_peered":"2024-09-15T23:36:29.454752+0000","last_clean":"2024-09-15T23:36:29.454752+0000","last_became_active":"2024-09-15T23:36:22.556929+0000","last_became_peered":"2024-09-15T23:36:22.556929+0000","last_unstale":"2024-09-15T23:36:29.454752+0000","last_undegraded":"2024-09-15T23:36:29.454752+0000","last_fullsized":"2024-09-15T23:36:29.454752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:57.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024486+0000","last_change":"2024-09-15T23:36:27.605955+0000","last_active":"2024-09-15T23:36:30.024486+0000","last_peered":"2024-09-15T23:36:30.024486+0000","last_clean":"2024-09-15T23:36:30.024486+0000","last_became_active":"2024-09-15T23:36:27.605521+0000","last_became_peered":"2024-09-15T23:36:27.605521+0000","last_unstale":"2024-09-15T23:36:30.024486+0000","last_undegraded":"2024-09-15T23:36:30.024486+0000","last_fullsized":"2024-09-15T23:36:30.024486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:23.776974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024511+0000","last_change":"2024-09-15T23:36:23.583781+0000","last_active":"2024-09-15T23:36:30.024511+0000","last_peered":"2024-09-15T23:36:30.024511+0000","last_clean":"2024-09-15T23:36:30.024511+0000","last_became_active":"2024-09-15T23:36:23.583475+0000","last_became_peered":"2024-09-15T23:36:23.583475+0000","last_unstale":"2024-09-15T23:36:30.024511+0000","last_undegraded":"2024-09-15T23:36:30.024511+0000","last_fullsized":"2024-09-15T23:36:30.024511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:23.983256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024984+0000","last_change":"2024-09-15T23:36:22.569943+0000","last_active":"2024-09-15T23:36:30.024984+0000","last_peered":"2024-09-15T23:36:30.024984+0000","last_clean":"2024-09-15T23:36:30.024984+0000","last_became_active":"2024-09-15T23:36:22.568617+0000","last_became_peered":"2024-09-15T23:36:22.568617+0000","last_unstale":"2024-09-15T23:36:30.024984+0000","last_undegraded":"2024-09-15T23:36:30.024984+0000","last_fullsized":"2024-09-15T23:36:30.024984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:44.229130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454281+0000","last_change":"2024-09-15T23:36:27.604270+0000","last_active":"2024-09-15T23:36:29.454281+0000","last_peered":"2024-09-15T23:36:29.454281+0000","last_clean":"2024-09-15T23:36:29.454281+0000","last_became_active":"2024-09-15T23:36:27.603416+0000","last_became_peered":"2024-09-15T23:36:27.603416+0000","last_unstale":"2024-09-15T23:36:29.454281+0000","last_undegraded":"2024-09-15T23:36:29.454281+0000","last_fullsized":"2024-09-15T23:36:29.454281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:12.747026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024835+0000","last_change":"2024-09-15T23:36:23.580585+0000","last_active":"2024-09-15T23:36:30.024835+0000","last_peered":"2024-09-15T23:36:30.024835+0000","last_clean":"2024-09-15T23:36:30.024835+0000","last_became_active":"2024-09-15T23:36:23.579003+0000","last_became_peered":"2024-09-15T23:36:23.579003+0000","last_unstale":"2024-09-15T23:36:30.024835+0000","last_undegraded":"2024-09-15T23:36:30.024835+0000","last_fullsized":"2024-09-15T23:36:30.024835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:24.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025091+0000","last_change":"2024-09-15T23:36:22.557388+0000","last_active":"2024-09-15T23:36:30.025091+0000","last_peered":"2024-09-15T23:36:30.025091+0000","last_clean":"2024-09-15T23:36:30.025091+0000","last_became_active":"2024-09-15T23:36:22.556138+0000","last_became_peered":"2024-09-15T23:36:22.556138+0000","last_unstale":"2024-09-15T23:36:30.025091+0000","last_undegraded":"2024-09-15T23:36:30.025091+0000","last_fullsized":"2024-09-15T23:36:30.025091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623557+0000","last_change":"2024-09-15T23:36:27.600449+0000","last_active":"2024-09-15T23:36:28.623557+0000","last_peered":"2024-09-15T23:36:28.623557+0000","last_clean":"2024-09-15T23:36:28.623557+0000","last_became_active":"2024-09-15T23:36:27.599937+0000","last_became_peered":"2024-09-15T23:36:27.599937+0000","last_unstale":"2024-09-15T23:36:28.623557+0000","last_undegraded":"2024-09-15T23:36:28.623557+0000","last_fullsized":"2024-09-15T23:36:28.623557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:13.541381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623785+0000","last_change":"2024-09-15T23:36:27.607667+0000","last_active":"2024-09-15T23:36:28.623785+0000","last_peered":"2024-09-15T23:36:28.623785+0000","last_clean":"2024-09-15T23:36:28.623785+0000","last_became_active":"2024-09-15T23:36:27.607227+0000","last_became_peered":"2024-09-15T23:36:27.607227+0000","last_unstale":"2024-09-15T23:36:28.623785+0000","last_undegraded":"2024-09-15T23:36:28.623785+0000","last_fullsized":"2024-09-15T23:36:28.623785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:33.572386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623619+0000","last_change":"2024-09-15T23:36:23.579014+0000","last_active":"2024-09-15T23:36:28.623619+0000","last_peered":"2024-09-15T23:36:28.623619+0000","last_clean":"2024-09-15T23:36:28.623619+0000","last_became_active":"2024-09-15T23:36:23.578709+0000","last_became_peered":"2024-09-15T23:36:23.578709+0000","last_unstale":"2024-09-15T23:36:28.623619+0000","last_undegraded":"2024-09-15T23:36:28.623619+0000","last_fullsized":"2024-09-15T23:36:28.623619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:07.005224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622050+0000","last_change":"2024-09-15T23:36:22.556649+0000","last_active":"2024-09-15T23:36:28.622050+0000","last_peered":"2024-09-15T23:36:28.622050+0000","last_clean":"2024-09-15T23:36:28.622050+0000","last_became_active":"2024-09-15T23:36:22.556332+0000","last_became_peered":"2024-09-15T23:36:22.556332+0000","last_unstale":"2024-09-15T23:36:28.622050+0000","last_undegraded":"2024-09-15T23:36:28.622050+0000","last_fullsized":"2024-09-15T23:36:28.622050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:46.855766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623003+0000","last_change":"2024-09-15T23:36:27.612447+0000","last_active":"2024-09-15T23:36:28.623003+0000","last_peered":"2024-09-15T23:36:28.623003+0000","last_clean":"2024-09-15T23:36:28.623003+0000","last_became_active":"2024-09-15T23:36:27.612007+0000","last_became_peered":"2024-09-15T23:36:27.612007+0000","last_unstale":"2024-09-15T23:36:28.623003+0000","last_undegraded":"2024-09-15T23:36:28.623003+0000","last_fullsized":"2024-09-15T23:36:28.623003+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:15.233014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023818+0000","last_change":"2024-09-15T23:36:23.592928+0000","last_active":"2024-09-15T23:36:30.023818+0000","last_peered":"2024-09-15T23:36:30.023818+0000","last_clean":"2024-09-15T23:36:30.023818+0000","last_became_active":"2024-09-15T23:36:23.592641+0000","last_became_peered":"2024-09-15T23:36:23.592641+0000","last_unstale":"2024-09-15T23:36:30.023818+0000","last_undegraded":"2024-09-15T23:36:30.023818+0000","last_fullsized":"2024-09-15T23:36:30.023818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:02.943531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622171+0000","last_change":"2024-09-15T23:36:22.569415+0000","last_active":"2024-09-15T23:36:28.622171+0000","last_peered":"2024-09-15T23:36:28.622171+0000","last_clean":"2024-09-15T23:36:28.622171+0000","last_became_active":"2024-09-15T23:36:22.569105+0000","last_became_peered":"2024-09-15T23:36:22.569105+0000","last_unstale":"2024-09-15T23:36:28.622171+0000","last_undegraded":"2024-09-15T23:36:28.622171+0000","last_fullsized":"2024-09-15T23:36:28.622171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622420+0000","last_change":"2024-09-15T23:36:22.569714+0000","last_active":"2024-09-15T23:36:28.622420+0000","last_peered":"2024-09-15T23:36:28.622420+0000","last_clean":"2024-09-15T23:36:28.622420+0000","last_became_active":"2024-09-15T23:36:22.569371+0000","last_became_peered":"2024-09-15T23:36:22.569371+0000","last_unstale":"2024-09-15T23:36:28.622420+0000","last_undegraded":"2024-09-15T23:36:28.622420+0000","last_fullsized":"2024-09-15T23:36:28.622420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:19.167117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622958+0000","last_change":"2024-09-15T23:36:23.591060+0000","last_active":"2024-09-15T23:36:28.622958+0000","last_peered":"2024-09-15T23:36:28.622958+0000","last_clean":"2024-09-15T23:36:28.622958+0000","last_became_active":"2024-09-15T23:36:23.590748+0000","last_became_peered":"2024-09-15T23:36:23.590748+0000","last_unstale":"2024-09-15T23:36:28.622958+0000","last_undegraded":"2024-09-15T23:36:28.622958+0000","last_fullsized":"2024-09-15T23:36:28.622958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:56.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624452+0000","last_change":"2024-09-15T23:36:27.602410+0000","last_active":"2024-09-15T23:36:28.624452+0000","last_peered":"2024-09-15T23:36:28.624452+0000","last_clean":"2024-09-15T23:36:28.624452+0000","last_became_active":"2024-09-15T23:36:27.602009+0000","last_became_peered":"2024-09-15T23:36:27.602009+0000","last_unstale":"2024-09-15T23:36:28.624452+0000","last_undegraded":"2024-09-15T23:36:28.624452+0000","last_fullsized":"2024-09-15T23:36:28.624452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:38.882900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024165+0000","last_change":"2024-09-15T23:36:22.562409+0000","last_active":"2024-09-15T23:36:30.024165+0000","last_peered":"2024-09-15T23:36:30.024165+0000","last_clean":"2024-09-15T23:36:30.024165+0000","last_became_active":"2024-09-15T23:36:22.561579+0000","last_became_peered":"2024-09-15T23:36:22.561579+0000","last_unstale":"2024-09-15T23:36:30.024165+0000","last_undegraded":"2024-09-15T23:36:30.024165+0000","last_fullsized":"2024-09-15T23:36:30.024165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:46.879331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023841+0000","last_change":"2024-09-15T23:36:23.584956+0000","last_active":"2024-09-15T23:36:30.023841+0000","last_peered":"2024-09-15T23:36:30.023841+0000","last_clean":"2024-09-15T23:36:30.023841+0000","last_became_active":"2024-09-15T23:36:23.584680+0000","last_became_peered":"2024-09-15T23:36:23.584680+0000","last_unstale":"2024-09-15T23:36:30.023841+0000","last_undegraded":"2024-09-15T23:36:30.023841+0000","last_fullsized":"2024-09-15T23:36:30.023841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454641+0000","last_change":"2024-09-15T23:36:27.603930+0000","last_active":"2024-09-15T23:36:29.454641+0000","last_peered":"2024-09-15T23:36:29.454641+0000","last_clean":"2024-09-15T23:36:29.454641+0000","last_became_active":"2024-09-15T23:36:27.603486+0000","last_became_peered":"2024-09-15T23:36:27.603486+0000","last_unstale":"2024-09-15T23:36:29.454641+0000","last_undegraded":"2024-09-15T23:36:29.454641+0000","last_fullsized":"2024-09-15T23:36:29.454641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:41.146991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024615+0000","last_change":"2024-09-15T23:36:23.590531+0000","last_active":"2024-09-15T23:36:30.024615+0000","last_peered":"2024-09-15T23:36:30.024615+0000","last_clean":"2024-09-15T23:36:30.024615+0000","last_became_active":"2024-09-15T23:36:23.590278+0000","last_became_peered":"2024-09-15T23:36:23.590278+0000","last_unstale":"2024-09-15T23:36:30.024615+0000","last_undegraded":"2024-09-15T23:36:30.024615+0000","last_fullsized":"2024-09-15T23:36:30.024615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454276+0000","last_change":"2024-09-15T23:36:23.580382+0000","last_active":"2024-09-15T23:36:29.454276+0000","last_peered":"2024-09-15T23:36:29.454276+0000","last_clean":"2024-09-15T23:36:29.454276+0000","last_became_active":"2024-09-15T23:36:23.580096+0000","last_became_peered":"2024-09-15T23:36:23.580096+0000","last_unstale":"2024-09-15T23:36:29.454276+0000","last_undegraded":"2024-09-15T23:36:29.454276+0000","last_fullsized":"2024-09-15T23:36:29.454276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622200+0000","last_change":"2024-09-15T23:36:23.587488+0000","last_active":"2024-09-15T23:36:28.622200+0000","last_peered":"2024-09-15T23:36:28.622200+0000","last_clean":"2024-09-15T23:36:28.622200+0000","last_became_active":"2024-09-15T23:36:23.587199+0000","last_became_peered":"2024-09-15T23:36:23.587199+0000","last_unstale":"2024-09-15T23:36:28.622200+0000","last_undegraded":"2024-09-15T23:36:28.622200+0000","last_fullsized":"2024-09-15T23:36:28.622200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:41.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023999+0000","last_change":"2024-09-15T23:36:23.583971+0000","last_active":"2024-09-15T23:36:30.023999+0000","last_peered":"2024-09-15T23:36:30.023999+0000","last_clean":"2024-09-15T23:36:30.023999+0000","last_became_active":"2024-09-15T23:36:23.583670+0000","last_became_peered":"2024-09-15T23:36:23.583670+0000","last_unstale":"2024-09-15T23:36:30.023999+0000","last_undegraded":"2024-09-15T23:36:30.023999+0000","last_fullsized":"2024-09-15T23:36:30.023999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:56.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622689+0000","last_change":"2024-09-15T23:36:23.598604+0000","last_active":"2024-09-15T23:36:28.622689+0000","last_peered":"2024-09-15T23:36:28.622689+0000","last_clean":"2024-09-15T23:36:28.622689+0000","last_became_active":"2024-09-15T23:36:23.598283+0000","last_became_peered":"2024-09-15T23:36:23.598283+0000","last_unstale":"2024-09-15T23:36:28.622689+0000","last_undegraded":"2024-09-15T23:36:28.622689+0000","last_fullsized":"2024-09-15T23:36:28.622689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:46.930120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622563+0000","last_change":"2024-09-15T23:36:23.588957+0000","last_active":"2024-09-15T23:36:28.622563+0000","last_peered":"2024-09-15T23:36:28.622563+0000","last_clean":"2024-09-15T23:36:28.622563+0000","last_became_active":"2024-09-15T23:36:23.588651+0000","last_became_peered":"2024-09-15T23:36:23.588651+0000","last_unstale":"2024-09-15T23:36:28.622563+0000","last_undegraded":"2024-09-15T23:36:28.622563+0000","last_fullsized":"2024-09-15T23:36:28.622563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:05.334429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024765+0000","last_change":"2024-09-15T23:36:23.579681+0000","last_active":"2024-09-15T23:36:30.024765+0000","last_peered":"2024-09-15T23:36:30.024765+0000","last_clean":"2024-09-15T23:36:30.024765+0000","last_became_active":"2024-09-15T23:36:23.579299+0000","last_became_peered":"2024-09-15T23:36:23.579299+0000","last_unstale":"2024-09-15T23:36:30.024765+0000","last_undegraded":"2024-09-15T23:36:30.024765+0000","last_fullsized":"2024-09-15T23:36:30.024765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:19.006869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024346+0000","last_change":"2024-09-15T23:36:23.587478+0000","last_active":"2024-09-15T23:36:30.024346+0000","last_peered":"2024-09-15T23:36:30.024346+0000","last_clean":"2024-09-15T23:36:30.024346+0000","last_became_active":"2024-09-15T23:36:23.586993+0000","last_became_peered":"2024-09-15T23:36:23.586993+0000","last_unstale":"2024-09-15T23:36:30.024346+0000","last_undegraded":"2024-09-15T23:36:30.024346+0000","last_fullsized":"2024-09-15T23:36:30.024346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:11:50.246113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622283+0000","last_change":"2024-09-15T23:36:23.594827+0000","last_active":"2024-09-15T23:36:28.622283+0000","last_peered":"2024-09-15T23:36:28.622283+0000","last_clean":"2024-09-15T23:36:28.622283+0000","last_became_active":"2024-09-15T23:36:23.594542+0000","last_became_peered":"2024-09-15T23:36:23.594542+0000","last_unstale":"2024-09-15T23:36:28.622283+0000","last_undegraded":"2024-09-15T23:36:28.622283+0000","last_fullsized":"2024-09-15T23:36:28.622283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.872038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023772+0000","last_change":"2024-09-15T23:36:23.588492+0000","last_active":"2024-09-15T23:36:30.023772+0000","last_peered":"2024-09-15T23:36:30.023772+0000","last_clean":"2024-09-15T23:36:30.023772+0000","last_became_active":"2024-09-15T23:36:23.588154+0000","last_became_peered":"2024-09-15T23:36:23.588154+0000","last_unstale":"2024-09-15T23:36:30.023772+0000","last_undegraded":"2024-09-15T23:36:30.023772+0000","last_fullsized":"2024-09-15T23:36:30.023772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:50:24.116022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023752+0000","last_change":"2024-09-15T23:36:23.584149+0000","last_active":"2024-09-15T23:36:30.023752+0000","last_peered":"2024-09-15T23:36:30.023752+0000","last_clean":"2024-09-15T23:36:30.023752+0000","last_became_active":"2024-09-15T23:36:23.583864+0000","last_became_peered":"2024-09-15T23:36:23.583864+0000","last_unstale":"2024-09-15T23:36:30.023752+0000","last_undegraded":"2024-09-15T23:36:30.023752+0000","last_fullsized":"2024-09-15T23:36:30.023752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:09.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622823+0000","last_change":"2024-09-15T23:36:23.581142+0000","last_active":"2024-09-15T23:36:28.622823+0000","last_peered":"2024-09-15T23:36:28.622823+0000","last_clean":"2024-09-15T23:36:28.622823+0000","last_became_active":"2024-09-15T23:36:23.580551+0000","last_became_peered":"2024-09-15T23:36:23.580551+0000","last_unstale":"2024-09-15T23:36:28.622823+0000","last_undegraded":"2024-09-15T23:36:28.622823+0000","last_fullsized":"2024-09-15T23:36:28.622823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:44.358074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025231+0000","last_change":"2024-09-15T23:36:23.596339+0000","last_active":"2024-09-15T23:36:30.025231+0000","last_peered":"2024-09-15T23:36:30.025231+0000","last_clean":"2024-09-15T23:36:30.025231+0000","last_became_active":"2024-09-15T23:36:23.590862+0000","last_became_peered":"2024-09-15T23:36:23.590862+0000","last_unstale":"2024-09-15T23:36:30.025231+0000","last_undegraded":"2024-09-15T23:36:30.025231+0000","last_fullsized":"2024-09-15T23:36:30.025231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:06.543007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623049+0000","last_change":"2024-09-15T23:36:23.589509+0000","last_active":"2024-09-15T23:36:28.623049+0000","last_peered":"2024-09-15T23:36:28.623049+0000","last_clean":"2024-09-15T23:36:28.623049+0000","last_became_active":"2024-09-15T23:36:23.589201+0000","last_became_peered":"2024-09-15T23:36:23.589201+0000","last_unstale":"2024-09-15T23:36:28.623049+0000","last_undegraded":"2024-09-15T23:36:28.623049+0000","last_fullsized":"2024-09-15T23:36:28.623049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:02.514262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454968+0000","last_change":"2024-09-15T23:36:23.582505+0000","last_active":"2024-09-15T23:36:29.454968+0000","last_peered":"2024-09-15T23:36:29.454968+0000","last_clean":"2024-09-15T23:36:29.454968+0000","last_became_active":"2024-09-15T23:36:23.582210+0000","last_became_peered":"2024-09-15T23:36:23.582210+0000","last_unstale":"2024-09-15T23:36:29.454968+0000","last_undegraded":"2024-09-15T23:36:29.454968+0000","last_fullsized":"2024-09-15T23:36:29.454968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623156+0000","last_change":"2024-09-15T23:36:23.590697+0000","last_active":"2024-09-15T23:36:28.623156+0000","last_peered":"2024-09-15T23:36:28.623156+0000","last_clean":"2024-09-15T23:36:28.623156+0000","last_became_active":"2024-09-15T23:36:23.590399+0000","last_became_peered":"2024-09-15T23:36:23.590399+0000","last_unstale":"2024-09-15T23:36:28.623156+0000","last_undegraded":"2024-09-15T23:36:28.623156+0000","last_fullsized":"2024-09-15T23:36:28.623156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.804071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621897+0000","last_change":"2024-09-15T23:36:23.583813+0000","last_active":"2024-09-15T23:36:28.621897+0000","last_peered":"2024-09-15T23:36:28.621897+0000","last_clean":"2024-09-15T23:36:28.621897+0000","last_became_active":"2024-09-15T23:36:23.583516+0000","last_became_peered":"2024-09-15T23:36:23.583516+0000","last_unstale":"2024-09-15T23:36:28.621897+0000","last_undegraded":"2024-09-15T23:36:28.621897+0000","last_fullsized":"2024-09-15T23:36:28.621897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:40.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023592+0000","last_change":"2024-09-15T23:36:23.587846+0000","last_active":"2024-09-15T23:36:30.023592+0000","last_peered":"2024-09-15T23:36:30.023592+0000","last_clean":"2024-09-15T23:36:30.023592+0000","last_became_active":"2024-09-15T23:36:23.587540+0000","last_became_peered":"2024-09-15T23:36:23.587540+0000","last_unstale":"2024-09-15T23:36:30.023592+0000","last_undegraded":"2024-09-15T23:36:30.023592+0000","last_fullsized":"2024-09-15T23:36:30.023592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:31.212365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621934+0000","last_change":"2024-09-15T23:36:23.589402+0000","last_active":"2024-09-15T23:36:28.621934+0000","last_peered":"2024-09-15T23:36:28.621934+0000","last_clean":"2024-09-15T23:36:28.621934+0000","last_became_active":"2024-09-15T23:36:23.589096+0000","last_became_peered":"2024-09-15T23:36:23.589096+0000","last_unstale":"2024-09-15T23:36:28.621934+0000","last_undegraded":"2024-09-15T23:36:28.621934+0000","last_fullsized":"2024-09-15T23:36:28.621934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:19.006869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621955+0000","last_change":"2024-09-15T23:36:23.598740+0000","last_active":"2024-09-15T23:36:28.621955+0000","last_peered":"2024-09-15T23:36:28.621955+0000","last_clean":"2024-09-15T23:36:28.621955+0000","last_became_active":"2024-09-15T23:36:23.598307+0000","last_became_peered":"2024-09-15T23:36:23.598307+0000","last_unstale":"2024-09-15T23:36:28.621955+0000","last_undegraded":"2024-09-15T23:36:28.621955+0000","last_fullsized":"2024-09-15T23:36:28.621955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:24.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025141+0000","last_change":"2024-09-15T23:36:23.591784+0000","last_active":"2024-09-15T23:36:30.025141+0000","last_peered":"2024-09-15T23:36:30.025141+0000","last_clean":"2024-09-15T23:36:30.025141+0000","last_became_active":"2024-09-15T23:36:23.591022+0000","last_became_peered":"2024-09-15T23:36:23.591022+0000","last_unstale":"2024-09-15T23:36:30.025141+0000","last_undegraded":"2024-09-15T23:36:30.025141+0000","last_fullsized":"2024-09-15T23:36:30.025141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:23.323481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025097+0000","last_change":"2024-09-15T23:36:23.586049+0000","last_active":"2024-09-15T23:36:30.025097+0000","last_peered":"2024-09-15T23:36:30.025097+0000","last_clean":"2024-09-15T23:36:30.025097+0000","last_became_active":"2024-09-15T23:36:23.585679+0000","last_became_peered":"2024-09-15T23:36:23.585679+0000","last_unstale":"2024-09-15T23:36:30.025097+0000","last_undegraded":"2024-09-15T23:36:30.025097+0000","last_fullsized":"2024-09-15T23:36:30.025097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:39.610334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024825+0000","last_change":"2024-09-15T23:36:23.595842+0000","last_active":"2024-09-15T23:36:30.024825+0000","last_peered":"2024-09-15T23:36:30.024825+0000","last_clean":"2024-09-15T23:36:30.024825+0000","last_became_active":"2024-09-15T23:36:23.595526+0000","last_became_peered":"2024-09-15T23:36:23.595526+0000","last_unstale":"2024-09-15T23:36:30.024825+0000","last_undegraded":"2024-09-15T23:36:30.024825+0000","last_fullsized":"2024-09-15T23:36:30.024825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:37.109784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025005+0000","last_change":"2024-09-15T23:36:23.572643+0000","last_active":"2024-09-15T23:36:30.025005+0000","last_peered":"2024-09-15T23:36:30.025005+0000","last_clean":"2024-09-15T23:36:30.025005+0000","last_became_active":"2024-09-15T23:36:23.572340+0000","last_became_peered":"2024-09-15T23:36:23.572340+0000","last_unstale":"2024-09-15T23:36:30.025005+0000","last_undegraded":"2024-09-15T23:36:30.025005+0000","last_fullsized":"2024-09-15T23:36:30.025005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023623+0000","last_change":"2024-09-15T23:36:23.592206+0000","last_active":"2024-09-15T23:36:30.023623+0000","last_peered":"2024-09-15T23:36:30.023623+0000","last_clean":"2024-09-15T23:36:30.023623+0000","last_became_active":"2024-09-15T23:36:23.590955+0000","last_became_peered":"2024-09-15T23:36:23.590955+0000","last_unstale":"2024-09-15T23:36:30.023623+0000","last_undegraded":"2024-09-15T23:36:30.023623+0000","last_fullsized":"2024-09-15T23:36:30.023623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:17.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023548+0000","last_change":"2024-09-15T23:36:23.594637+0000","last_active":"2024-09-15T23:36:30.023548+0000","last_peered":"2024-09-15T23:36:30.023548+0000","last_clean":"2024-09-15T23:36:30.023548+0000","last_became_active":"2024-09-15T23:36:23.594333+0000","last_became_peered":"2024-09-15T23:36:23.594333+0000","last_unstale":"2024-09-15T23:36:30.023548+0000","last_undegraded":"2024-09-15T23:36:30.023548+0000","last_fullsized":"2024-09-15T23:36:30.023548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:51.033435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024411+0000","last_change":"2024-09-15T23:36:23.592276+0000","last_active":"2024-09-15T23:36:30.024411+0000","last_peered":"2024-09-15T23:36:30.024411+0000","last_clean":"2024-09-15T23:36:30.024411+0000","last_became_active":"2024-09-15T23:36:23.591988+0000","last_became_peered":"2024-09-15T23:36:23.591988+0000","last_unstale":"2024-09-15T23:36:30.024411+0000","last_undegraded":"2024-09-15T23:36:30.024411+0000","last_fullsized":"2024-09-15T23:36:30.024411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024046+0000","last_change":"2024-09-15T23:36:23.596925+0000","last_active":"2024-09-15T23:36:30.024046+0000","last_peered":"2024-09-15T23:36:30.024046+0000","last_clean":"2024-09-15T23:36:30.024046+0000","last_became_active":"2024-09-15T23:36:23.596610+0000","last_became_peered":"2024-09-15T23:36:23.596610+0000","last_unstale":"2024-09-15T23:36:30.024046+0000","last_undegraded":"2024-09-15T23:36:30.024046+0000","last_fullsized":"2024-09-15T23:36:30.024046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:56.256643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023789+0000","last_change":"2024-09-15T23:36:23.591422+0000","last_active":"2024-09-15T23:36:30.023789+0000","last_peered":"2024-09-15T23:36:30.023789+0000","last_clean":"2024-09-15T23:36:30.023789+0000","last_became_active":"2024-09-15T23:36:23.590380+0000","last_became_peered":"2024-09-15T23:36:23.590380+0000","last_unstale":"2024-09-15T23:36:30.023789+0000","last_undegraded":"2024-09-15T23:36:30.023789+0000","last_fullsized":"2024-09-15T23:36:30.023789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:20.169043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024050+0000","last_change":"2024-09-15T23:36:23.592150+0000","last_active":"2024-09-15T23:36:30.024050+0000","last_peered":"2024-09-15T23:36:30.024050+0000","last_clean":"2024-09-15T23:36:30.024050+0000","last_became_active":"2024-09-15T23:36:23.591205+0000","last_became_peered":"2024-09-15T23:36:23.591205+0000","last_unstale":"2024-09-15T23:36:30.024050+0000","last_undegraded":"2024-09-15T23:36:30.024050+0000","last_fullsized":"2024-09-15T23:36:30.024050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:50.950584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024247+0000","last_change":"2024-09-15T23:36:23.593241+0000","last_active":"2024-09-15T23:36:30.024247+0000","last_peered":"2024-09-15T23:36:30.024247+0000","last_clean":"2024-09-15T23:36:30.024247+0000","last_became_active":"2024-09-15T23:36:23.591733+0000","last_became_peered":"2024-09-15T23:36:23.591733+0000","last_unstale":"2024-09-15T23:36:30.024247+0000","last_undegraded":"2024-09-15T23:36:30.024247+0000","last_fullsized":"2024-09-15T23:36:30.024247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:23.692197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024357+0000","last_change":"2024-09-15T23:36:23.596045+0000","last_active":"2024-09-15T23:36:30.024357+0000","last_peered":"2024-09-15T23:36:30.024357+0000","last_clean":"2024-09-15T23:36:30.024357+0000","last_became_active":"2024-09-15T23:36:23.595078+0000","last_became_peered":"2024-09-15T23:36:23.595078+0000","last_unstale":"2024-09-15T23:36:30.024357+0000","last_undegraded":"2024-09-15T23:36:30.024357+0000","last_fullsized":"2024-09-15T23:36:30.024357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:40.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":262144,"data_stored":103393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":270336,"data_stored":106655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":856064,"data_stored":693249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":102867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":102935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":258048,"data_stored":102811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":860160,"data_stored":693271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":107327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:36:43.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:36:44.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.005+0000 7f440d0e4640 1 Processor -- start 2024-09-15T23:36:44.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.005+0000 7f440d0e4640 1 -- start start 2024-09-15T23:36:44.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.006+0000 7f440d0e4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f44080d0f90 0x7f44080d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.006+0000 7f440d0e4640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44080d0a30 con 0x7f440815e840 2024-09-15T23:36:44.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.006+0000 7f440d0e4640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f440806dd80 con 0x7f44080d52e0 2024-09-15T23:36:44.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.006+0000 7f440d0e4640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f44080cfe90 con 0x7f44080d0f90 2024-09-15T23:36:44.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.006+0000 7f4406d9d640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f440815e840 0x7f44080d0320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44632/0 (socket says 172.21.15.7:44632) 2024-09-15T23:36:44.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.006+0000 7f4406d9d640 1 -- 172.21.15.7:0/2584802979 learned_addr learned my addr 172.21.15.7:0/2584802979 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:44.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f440759e640 1 --2- 172.21.15.7:0/2584802979 >> v2:172.21.15.18:3300/0 conn(0x7f44080d0f90 0x7f44080d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2561022087 0 0) 0x7f44080d0a30 con 0x7f440815e840 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43f0003520 con 0x7f440815e840 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3858613636 0 0) 0x7f43f0003520 con 0x7f440815e840 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 >> v2:172.21.15.18:3300/0 conn(0x7f44080d0f90 msgr2=0x7f44080d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 --2- 172.21.15.7:0/2584802979 >> v2:172.21.15.18:3300/0 conn(0x7f44080d0f90 0x7f44080d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 >> v1:172.21.15.18:6789/0 conn(0x7f44080d52e0 legacy=0x7f440806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44080cf560 con 0x7f440815e840 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f43f80019f0 con 0x7f440815e840 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 >> v1:172.21.15.7:6789/0 conn(0x7f440815e840 legacy=0x7f44080d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f00039b0 0x7f43f0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f0004800 0x7f43f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f00051c0 0x7f43f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44080cfe90 con 0x7f43f0004800 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f43f000c090 con 0x7f43f00039b0 2024-09-15T23:36:44.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f43f000c290 con 0x7f43f00051c0 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f440659c640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f0004800 0x7f43f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f4406d9d640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f00039b0 0x7f43f0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f440759e640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f00051c0 0x7f43f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f440659c640 1 -- 172.21.15.7:0/2584802979 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f00051c0 msgr2=0x7f43f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f440659c640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f00051c0 0x7f43f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f440659c640 1 -- 172.21.15.7:0/2584802979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f00039b0 msgr2=0x7f43f0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f440659c640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f00039b0 0x7f43f0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f440659c640 1 -- 172.21.15.7:0/2584802979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f0003520 con 0x7f43f0004800 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.007+0000 7f440759e640 1 -- 172.21.15.7:0/2584802979 reap_dead start 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.008+0000 7f440659c640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f0004800 0x7f43f0004c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43fc001770 tx=0x7f43f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcef9673f7424ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.008+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0013c0 con 0x7f43f0004800 2024-09-15T23:36:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.008+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43fc0052d0 con 0x7f43f0004800 2024-09-15T23:36:44.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.008+0000 7f4405d9b640 1 -- 172.21.15.7:0/2584802979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc02a8a0 con 0x7f43f0004800 2024-09-15T23:36:44.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.009+0000 7f440d0e4640 1 -- 172.21.15.7:0/2584802979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f0004800 msgr2=0x7f43f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.009+0000 7f440d0e4640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f0004800 0x7f43f0004c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43fc001770 tx=0x7f43f0005830 comp rx=0 tx=0).stop 2024-09-15T23:36:44.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.009+0000 7f440d0e4640 1 -- 172.21.15.7:0/2584802979 shutdown_connections 2024-09-15T23:36:44.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.009+0000 7f440d0e4640 1 --2- 172.21.15.7:0/2584802979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f0004800 0x7f43f0004c80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.009+0000 7f440d0e4640 1 -- 172.21.15.7:0/2584802979 >> 172.21.15.7:0/2584802979 conn(0x7f440808bfd0 msgr2=0x7f44080d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:44.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.009+0000 7f440d0e4640 1 -- 172.21.15.7:0/2584802979 shutdown_connections 2024-09-15T23:36:44.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.009+0000 7f440d0e4640 1 -- 172.21.15.7:0/2584802979 wait complete. 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.010+0000 7f440d0e4640 1 Processor -- start 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440d0e4640 1 -- start start 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440d0e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f0004800 0x7f44080b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440d0e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44080cb160 0x7f4408158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440d0e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44080cc030 0x7f44080cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440d0e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f440806e030 con 0x7f44080cb160 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440d0e4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f44080cfb20 con 0x7f44080cc030 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440d0e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f440815ec20 con 0x7f43f0004800 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44080cb160 0x7f4408158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44080cb160 0x7f4408158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48114/0 (socket says 172.21.15.7:48114) 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440659c640 1 -- 172.21.15.7:0/2677409736 learned_addr learned my addr 172.21.15.7:0/2677409736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f4406d9d640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f0004800 0x7f44080b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.011+0000 7f440759e640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44080cc030 0x7f44080cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f4406d9d640 1 -- 172.21.15.7:0/2677409736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44080cc030 msgr2=0x7f44080cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f4406d9d640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44080cc030 0x7f44080cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f4406d9d640 1 -- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44080cb160 msgr2=0x7f4408158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f4406d9d640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44080cb160 0x7f4408158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f4406d9d640 1 -- 172.21.15.7:0/2677409736 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44080b63c0 con 0x7f43f0004800 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f440659c640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44080cb160 0x7f4408158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f4406d9d640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f0004800 0x7f44080b5b00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f43f8007740 tx=0x7f43f8007c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f43ef7fe640 1 -- 172.21.15.7:0/2677409736 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8017020 con 0x7f43f0004800 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f43ef7fe640 1 -- 172.21.15.7:0/2677409736 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f800acd0 con 0x7f43f0004800 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44080b66a0 con 0x7f43f0004800 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f43ef7fe640 1 -- 172.21.15.7:0/2677409736 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f80135f0 con 0x7f43f0004800 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.012+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44080cbb10 con 0x7f43f0004800 2024-09-15T23:36:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.013+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44080d13c0 con 0x7f43f0004800 2024-09-15T23:36:44.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.014+0000 7f43ef7fe640 1 -- 172.21.15.7:0/2677409736 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f43f801a020 con 0x7f43f0004800 2024-09-15T23:36:44.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.014+0000 7f43ef7fe640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f43f406c2b0 0x7f43f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.014+0000 7f43ef7fe640 1 -- 172.21.15.7:0/2677409736 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f43f808bed0 con 0x7f43f0004800 2024-09-15T23:36:44.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.014+0000 7f440659c640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f43f406c2b0 0x7f43f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.015+0000 7f440659c640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f43f406c2b0 0x7f43f406e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43fc005040 tx=0x7f43fc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:44.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.019+0000 7f43ef7fe640 1 -- 172.21.15.7:0/2677409736 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f43f8021020 con 0x7f43f0004800 2024-09-15T23:36:44.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.191+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44080d1cb0 con 0x7f43f406c2b0 2024-09-15T23:36:44.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.199+0000 7f43ef7fe640 1 -- 172.21.15.7:0/2677409736 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7f44080d1cb0 con 0x7f43f406c2b0 2024-09-15T23:36:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:44.206 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:36:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f43f406c2b0 msgr2=0x7f43f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f43f406c2b0 0x7f43f406e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43fc005040 tx=0x7f43fc02e000 comp rx=0 tx=0).stop 2024-09-15T23:36:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f0004800 msgr2=0x7f44080b5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f0004800 0x7f44080b5b00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f43f8007740 tx=0x7f43f8007c00 comp rx=0 tx=0).stop 2024-09-15T23:36:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 shutdown_connections 2024-09-15T23:36:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44080cc030 0x7f44080cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44080cb160 0x7f4408158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f43f406c2b0 0x7f43f406e770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 --2- 172.21.15.7:0/2677409736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f43f0004800 0x7f44080b5b00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 >> 172.21.15.7:0/2677409736 conn(0x7f440808bfd0 msgr2=0x7f440815ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 shutdown_connections 2024-09-15T23:36:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.203+0000 7f440d0e4640 1 -- 172.21.15.7:0/2677409736 wait complete. 2024-09-15T23:36:44.215 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-15T23:36:43.815857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252752,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721968,"statfs":{"total":773094113280,"available":772835295232,"internally_reserved":0,"allocated":3317760,"data_stored":2012508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002861"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623447+0000","last_change":"2024-09-15T23:36:22.558162+0000","last_active":"2024-09-15T23:36:28.623447+0000","last_peered":"2024-09-15T23:36:28.623447+0000","last_clean":"2024-09-15T23:36:28.623447+0000","last_became_active":"2024-09-15T23:36:22.557752+0000","last_became_peered":"2024-09-15T23:36:22.557752+0000","last_unstale":"2024-09-15T23:36:28.623447+0000","last_undegraded":"2024-09-15T23:36:28.623447+0000","last_fullsized":"2024-09-15T23:36:28.623447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:17.806236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024436+0000","last_change":"2024-09-15T23:36:23.591313+0000","last_active":"2024-09-15T23:36:30.024436+0000","last_peered":"2024-09-15T23:36:30.024436+0000","last_clean":"2024-09-15T23:36:30.024436+0000","last_became_active":"2024-09-15T23:36:23.590422+0000","last_became_peered":"2024-09-15T23:36:23.590422+0000","last_unstale":"2024-09-15T23:36:30.024436+0000","last_undegraded":"2024-09-15T23:36:30.024436+0000","last_fullsized":"2024-09-15T23:36:30.024436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454338+0000","last_change":"2024-09-15T23:36:27.609388+0000","last_active":"2024-09-15T23:36:29.454338+0000","last_peered":"2024-09-15T23:36:29.454338+0000","last_clean":"2024-09-15T23:36:29.454338+0000","last_became_active":"2024-09-15T23:36:27.608999+0000","last_became_peered":"2024-09-15T23:36:27.608999+0000","last_unstale":"2024-09-15T23:36:29.454338+0000","last_undegraded":"2024-09-15T23:36:29.454338+0000","last_fullsized":"2024-09-15T23:36:29.454338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:30.088730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024564+0000","last_change":"2024-09-15T23:36:22.562097+0000","last_active":"2024-09-15T23:36:30.024564+0000","last_peered":"2024-09-15T23:36:30.024564+0000","last_clean":"2024-09-15T23:36:30.024564+0000","last_became_active":"2024-09-15T23:36:22.561356+0000","last_became_peered":"2024-09-15T23:36:22.561356+0000","last_unstale":"2024-09-15T23:36:30.024564+0000","last_undegraded":"2024-09-15T23:36:30.024564+0000","last_fullsized":"2024-09-15T23:36:30.024564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:46.482072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024461+0000","last_change":"2024-09-15T23:36:23.589236+0000","last_active":"2024-09-15T23:36:30.024461+0000","last_peered":"2024-09-15T23:36:30.024461+0000","last_clean":"2024-09-15T23:36:30.024461+0000","last_became_active":"2024-09-15T23:36:23.588873+0000","last_became_peered":"2024-09-15T23:36:23.588873+0000","last_unstale":"2024-09-15T23:36:30.024461+0000","last_undegraded":"2024-09-15T23:36:30.024461+0000","last_fullsized":"2024-09-15T23:36:30.024461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:18.209937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024371+0000","last_change":"2024-09-15T23:36:27.611526+0000","last_active":"2024-09-15T23:36:30.024371+0000","last_peered":"2024-09-15T23:36:30.024371+0000","last_clean":"2024-09-15T23:36:30.024371+0000","last_became_active":"2024-09-15T23:36:27.610759+0000","last_became_peered":"2024-09-15T23:36:27.610759+0000","last_unstale":"2024-09-15T23:36:30.024371+0000","last_undegraded":"2024-09-15T23:36:30.024371+0000","last_fullsized":"2024-09-15T23:36:30.024371+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:06.783968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025217+0000","last_change":"2024-09-15T23:36:22.568198+0000","last_active":"2024-09-15T23:36:30.025217+0000","last_peered":"2024-09-15T23:36:30.025217+0000","last_clean":"2024-09-15T23:36:30.025217+0000","last_became_active":"2024-09-15T23:36:22.567888+0000","last_became_peered":"2024-09-15T23:36:22.567888+0000","last_unstale":"2024-09-15T23:36:30.025217+0000","last_undegraded":"2024-09-15T23:36:30.025217+0000","last_fullsized":"2024-09-15T23:36:30.025217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:14:47.610942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024413+0000","last_change":"2024-09-15T23:36:23.593279+0000","last_active":"2024-09-15T23:36:30.024413+0000","last_peered":"2024-09-15T23:36:30.024413+0000","last_clean":"2024-09-15T23:36:30.024413+0000","last_became_active":"2024-09-15T23:36:23.592921+0000","last_became_peered":"2024-09-15T23:36:23.592921+0000","last_unstale":"2024-09-15T23:36:30.024413+0000","last_undegraded":"2024-09-15T23:36:30.024413+0000","last_fullsized":"2024-09-15T23:36:30.024413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:40.194400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024320+0000","last_change":"2024-09-15T23:36:27.612903+0000","last_active":"2024-09-15T23:36:30.024320+0000","last_peered":"2024-09-15T23:36:30.024320+0000","last_clean":"2024-09-15T23:36:30.024320+0000","last_became_active":"2024-09-15T23:36:27.612441+0000","last_became_peered":"2024-09-15T23:36:27.612441+0000","last_unstale":"2024-09-15T23:36:30.024320+0000","last_undegraded":"2024-09-15T23:36:30.024320+0000","last_fullsized":"2024-09-15T23:36:30.024320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:43.647541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025323+0000","last_change":"2024-09-15T23:36:22.566488+0000","last_active":"2024-09-15T23:36:30.025323+0000","last_peered":"2024-09-15T23:36:30.025323+0000","last_clean":"2024-09-15T23:36:30.025323+0000","last_became_active":"2024-09-15T23:36:22.566016+0000","last_became_peered":"2024-09-15T23:36:22.566016+0000","last_unstale":"2024-09-15T23:36:30.025323+0000","last_undegraded":"2024-09-15T23:36:30.025323+0000","last_fullsized":"2024-09-15T23:36:30.025323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:45.276407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622706+0000","last_change":"2024-09-15T23:36:23.590254+0000","last_active":"2024-09-15T23:36:28.622706+0000","last_peered":"2024-09-15T23:36:28.622706+0000","last_clean":"2024-09-15T23:36:28.622706+0000","last_became_active":"2024-09-15T23:36:23.589902+0000","last_became_peered":"2024-09-15T23:36:23.589902+0000","last_unstale":"2024-09-15T23:36:28.622706+0000","last_undegraded":"2024-09-15T23:36:28.622706+0000","last_fullsized":"2024-09-15T23:36:28.622706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:37.196308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024145+0000","last_change":"2024-09-15T23:36:27.603610+0000","last_active":"2024-09-15T23:36:30.024145+0000","last_peered":"2024-09-15T23:36:30.024145+0000","last_clean":"2024-09-15T23:36:30.024145+0000","last_became_active":"2024-09-15T23:36:27.603177+0000","last_became_peered":"2024-09-15T23:36:27.603177+0000","last_unstale":"2024-09-15T23:36:30.024145+0000","last_undegraded":"2024-09-15T23:36:30.024145+0000","last_fullsized":"2024-09-15T23:36:30.024145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:55.070642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622285+0000","last_change":"2024-09-15T23:36:23.576248+0000","last_active":"2024-09-15T23:36:28.622285+0000","last_peered":"2024-09-15T23:36:28.622285+0000","last_clean":"2024-09-15T23:36:28.622285+0000","last_became_active":"2024-09-15T23:36:23.575715+0000","last_became_peered":"2024-09-15T23:36:23.575715+0000","last_unstale":"2024-09-15T23:36:28.622285+0000","last_undegraded":"2024-09-15T23:36:28.622285+0000","last_fullsized":"2024-09-15T23:36:28.622285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:59.081263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622550+0000","last_change":"2024-09-15T23:36:22.555862+0000","last_active":"2024-09-15T23:36:28.622550+0000","last_peered":"2024-09-15T23:36:28.622550+0000","last_clean":"2024-09-15T23:36:28.622550+0000","last_became_active":"2024-09-15T23:36:22.555589+0000","last_became_peered":"2024-09-15T23:36:22.555589+0000","last_unstale":"2024-09-15T23:36:28.622550+0000","last_undegraded":"2024-09-15T23:36:28.622550+0000","last_fullsized":"2024-09-15T23:36:28.622550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:44.229130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622578+0000","last_change":"2024-09-15T23:36:27.602404+0000","last_active":"2024-09-15T23:36:28.622578+0000","last_peered":"2024-09-15T23:36:28.622578+0000","last_clean":"2024-09-15T23:36:28.622578+0000","last_became_active":"2024-09-15T23:36:27.601991+0000","last_became_peered":"2024-09-15T23:36:27.601991+0000","last_unstale":"2024-09-15T23:36:28.622578+0000","last_undegraded":"2024-09-15T23:36:28.622578+0000","last_fullsized":"2024-09-15T23:36:28.622578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:00.662926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454749+0000","last_change":"2024-09-15T23:36:23.591252+0000","last_active":"2024-09-15T23:36:29.454749+0000","last_peered":"2024-09-15T23:36:29.454749+0000","last_clean":"2024-09-15T23:36:29.454749+0000","last_became_active":"2024-09-15T23:36:23.590976+0000","last_became_peered":"2024-09-15T23:36:23.590976+0000","last_unstale":"2024-09-15T23:36:29.454749+0000","last_undegraded":"2024-09-15T23:36:29.454749+0000","last_fullsized":"2024-09-15T23:36:29.454749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622161+0000","last_change":"2024-09-15T23:36:22.560352+0000","last_active":"2024-09-15T23:36:28.622161+0000","last_peered":"2024-09-15T23:36:28.622161+0000","last_clean":"2024-09-15T23:36:28.622161+0000","last_became_active":"2024-09-15T23:36:22.560052+0000","last_became_peered":"2024-09-15T23:36:22.560052+0000","last_unstale":"2024-09-15T23:36:28.622161+0000","last_undegraded":"2024-09-15T23:36:28.622161+0000","last_fullsized":"2024-09-15T23:36:28.622161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023452+0000","last_change":"2024-09-15T23:36:27.598358+0000","last_active":"2024-09-15T23:36:30.023452+0000","last_peered":"2024-09-15T23:36:30.023452+0000","last_clean":"2024-09-15T23:36:30.023452+0000","last_became_active":"2024-09-15T23:36:27.597941+0000","last_became_peered":"2024-09-15T23:36:27.597941+0000","last_unstale":"2024-09-15T23:36:30.023452+0000","last_undegraded":"2024-09-15T23:36:30.023452+0000","last_fullsized":"2024-09-15T23:36:30.023452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:54.115527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622271+0000","last_change":"2024-09-15T23:36:23.581091+0000","last_active":"2024-09-15T23:36:28.622271+0000","last_peered":"2024-09-15T23:36:28.622271+0000","last_clean":"2024-09-15T23:36:28.622271+0000","last_became_active":"2024-09-15T23:36:23.580807+0000","last_became_peered":"2024-09-15T23:36:23.580807+0000","last_unstale":"2024-09-15T23:36:28.622271+0000","last_undegraded":"2024-09-15T23:36:28.622271+0000","last_fullsized":"2024-09-15T23:36:28.622271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:01.465502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623733+0000","last_change":"2024-09-15T23:36:22.569473+0000","last_active":"2024-09-15T23:36:28.623733+0000","last_peered":"2024-09-15T23:36:28.623733+0000","last_clean":"2024-09-15T23:36:28.623733+0000","last_became_active":"2024-09-15T23:36:22.569168+0000","last_became_peered":"2024-09-15T23:36:22.569168+0000","last_unstale":"2024-09-15T23:36:28.623733+0000","last_undegraded":"2024-09-15T23:36:28.623733+0000","last_fullsized":"2024-09-15T23:36:28.623733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:46.065174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454933+0000","last_change":"2024-09-15T23:36:27.605801+0000","last_active":"2024-09-15T23:36:29.454933+0000","last_peered":"2024-09-15T23:36:29.454933+0000","last_clean":"2024-09-15T23:36:29.454933+0000","last_became_active":"2024-09-15T23:36:27.605416+0000","last_became_peered":"2024-09-15T23:36:27.605416+0000","last_unstale":"2024-09-15T23:36:29.454933+0000","last_undegraded":"2024-09-15T23:36:29.454933+0000","last_fullsized":"2024-09-15T23:36:29.454933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:27.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622056+0000","last_change":"2024-09-15T23:36:23.587550+0000","last_active":"2024-09-15T23:36:28.622056+0000","last_peered":"2024-09-15T23:36:28.622056+0000","last_clean":"2024-09-15T23:36:28.622056+0000","last_became_active":"2024-09-15T23:36:23.587213+0000","last_became_peered":"2024-09-15T23:36:23.587213+0000","last_unstale":"2024-09-15T23:36:28.622056+0000","last_undegraded":"2024-09-15T23:36:28.622056+0000","last_fullsized":"2024-09-15T23:36:28.622056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:15.353430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023245+0000","last_change":"2024-09-15T23:36:22.569502+0000","last_active":"2024-09-15T23:36:30.023245+0000","last_peered":"2024-09-15T23:36:30.023245+0000","last_clean":"2024-09-15T23:36:30.023245+0000","last_became_active":"2024-09-15T23:36:22.567715+0000","last_became_peered":"2024-09-15T23:36:22.567715+0000","last_unstale":"2024-09-15T23:36:30.023245+0000","last_undegraded":"2024-09-15T23:36:30.023245+0000","last_fullsized":"2024-09-15T23:36:30.023245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024741+0000","last_change":"2024-09-15T23:36:27.612217+0000","last_active":"2024-09-15T23:36:30.024741+0000","last_peered":"2024-09-15T23:36:30.024741+0000","last_clean":"2024-09-15T23:36:30.024741+0000","last_became_active":"2024-09-15T23:36:27.611783+0000","last_became_peered":"2024-09-15T23:36:27.611783+0000","last_unstale":"2024-09-15T23:36:30.024741+0000","last_undegraded":"2024-09-15T23:36:30.024741+0000","last_fullsized":"2024-09-15T23:36:30.024741+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:45.020968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623852+0000","last_change":"2024-09-15T23:36:23.576178+0000","last_active":"2024-09-15T23:36:28.623852+0000","last_peered":"2024-09-15T23:36:28.623852+0000","last_clean":"2024-09-15T23:36:28.623852+0000","last_became_active":"2024-09-15T23:36:23.575823+0000","last_became_peered":"2024-09-15T23:36:23.575823+0000","last_unstale":"2024-09-15T23:36:28.623852+0000","last_undegraded":"2024-09-15T23:36:28.623852+0000","last_fullsized":"2024-09-15T23:36:28.623852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:17.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623244+0000","last_change":"2024-09-15T23:36:22.570936+0000","last_active":"2024-09-15T23:36:28.623244+0000","last_peered":"2024-09-15T23:36:28.623244+0000","last_clean":"2024-09-15T23:36:28.623244+0000","last_became_active":"2024-09-15T23:36:22.569461+0000","last_became_peered":"2024-09-15T23:36:22.569461+0000","last_unstale":"2024-09-15T23:36:28.623244+0000","last_undegraded":"2024-09-15T23:36:28.623244+0000","last_fullsized":"2024-09-15T23:36:28.623244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:38.263694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024707+0000","last_change":"2024-09-15T23:36:27.605878+0000","last_active":"2024-09-15T23:36:30.024707+0000","last_peered":"2024-09-15T23:36:30.024707+0000","last_clean":"2024-09-15T23:36:30.024707+0000","last_became_active":"2024-09-15T23:36:27.605448+0000","last_became_peered":"2024-09-15T23:36:27.605448+0000","last_unstale":"2024-09-15T23:36:30.024707+0000","last_undegraded":"2024-09-15T23:36:30.024707+0000","last_fullsized":"2024-09-15T23:36:30.024707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:00.293850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623345+0000","last_change":"2024-09-15T23:36:23.590802+0000","last_active":"2024-09-15T23:36:28.623345+0000","last_peered":"2024-09-15T23:36:28.623345+0000","last_clean":"2024-09-15T23:36:28.623345+0000","last_became_active":"2024-09-15T23:36:23.590505+0000","last_became_peered":"2024-09-15T23:36:23.590505+0000","last_unstale":"2024-09-15T23:36:28.623345+0000","last_undegraded":"2024-09-15T23:36:28.623345+0000","last_fullsized":"2024-09-15T23:36:28.623345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:50.950584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624097+0000","last_change":"2024-09-15T23:36:22.569675+0000","last_active":"2024-09-15T23:36:28.624097+0000","last_peered":"2024-09-15T23:36:28.624097+0000","last_clean":"2024-09-15T23:36:28.624097+0000","last_became_active":"2024-09-15T23:36:22.569386+0000","last_became_peered":"2024-09-15T23:36:22.569386+0000","last_unstale":"2024-09-15T23:36:28.624097+0000","last_undegraded":"2024-09-15T23:36:28.624097+0000","last_fullsized":"2024-09-15T23:36:28.624097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:00.420124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622250+0000","last_change":"2024-09-15T23:36:27.601084+0000","last_active":"2024-09-15T23:36:28.622250+0000","last_peered":"2024-09-15T23:36:28.622250+0000","last_clean":"2024-09-15T23:36:28.622250+0000","last_became_active":"2024-09-15T23:36:27.600671+0000","last_became_peered":"2024-09-15T23:36:27.600671+0000","last_unstale":"2024-09-15T23:36:28.622250+0000","last_undegraded":"2024-09-15T23:36:28.622250+0000","last_fullsized":"2024-09-15T23:36:28.622250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:28.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622694+0000","last_change":"2024-09-15T23:36:23.595285+0000","last_active":"2024-09-15T23:36:28.622694+0000","last_peered":"2024-09-15T23:36:28.622694+0000","last_clean":"2024-09-15T23:36:28.622694+0000","last_became_active":"2024-09-15T23:36:23.594596+0000","last_became_peered":"2024-09-15T23:36:23.594596+0000","last_unstale":"2024-09-15T23:36:28.622694+0000","last_undegraded":"2024-09-15T23:36:28.622694+0000","last_fullsized":"2024-09-15T23:36:28.622694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:39.265619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623245+0000","last_change":"2024-09-15T23:36:22.553052+0000","last_active":"2024-09-15T23:36:28.623245+0000","last_peered":"2024-09-15T23:36:28.623245+0000","last_clean":"2024-09-15T23:36:28.623245+0000","last_became_active":"2024-09-15T23:36:22.552766+0000","last_became_peered":"2024-09-15T23:36:22.552766+0000","last_unstale":"2024-09-15T23:36:28.623245+0000","last_undegraded":"2024-09-15T23:36:28.623245+0000","last_fullsized":"2024-09-15T23:36:28.623245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:45.928194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621909+0000","last_change":"2024-09-15T23:36:27.601501+0000","last_active":"2024-09-15T23:36:28.621909+0000","last_peered":"2024-09-15T23:36:28.621909+0000","last_clean":"2024-09-15T23:36:28.621909+0000","last_became_active":"2024-09-15T23:36:27.601071+0000","last_became_peered":"2024-09-15T23:36:27.601071+0000","last_unstale":"2024-09-15T23:36:28.621909+0000","last_undegraded":"2024-09-15T23:36:28.621909+0000","last_fullsized":"2024-09-15T23:36:28.621909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:03.241943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024193+0000","last_change":"2024-09-15T23:36:23.591415+0000","last_active":"2024-09-15T23:36:30.024193+0000","last_peered":"2024-09-15T23:36:30.024193+0000","last_clean":"2024-09-15T23:36:30.024193+0000","last_became_active":"2024-09-15T23:36:23.591132+0000","last_became_peered":"2024-09-15T23:36:23.591132+0000","last_unstale":"2024-09-15T23:36:30.024193+0000","last_undegraded":"2024-09-15T23:36:30.024193+0000","last_fullsized":"2024-09-15T23:36:30.024193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024034+0000","last_change":"2024-09-15T23:36:22.561640+0000","last_active":"2024-09-15T23:36:30.024034+0000","last_peered":"2024-09-15T23:36:30.024034+0000","last_clean":"2024-09-15T23:36:30.024034+0000","last_became_active":"2024-09-15T23:36:22.561218+0000","last_became_peered":"2024-09-15T23:36:22.561218+0000","last_unstale":"2024-09-15T23:36:30.024034+0000","last_undegraded":"2024-09-15T23:36:30.024034+0000","last_fullsized":"2024-09-15T23:36:30.024034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:06:31.226510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621863+0000","last_change":"2024-09-15T23:36:27.606889+0000","last_active":"2024-09-15T23:36:28.621863+0000","last_peered":"2024-09-15T23:36:28.621863+0000","last_clean":"2024-09-15T23:36:28.621863+0000","last_became_active":"2024-09-15T23:36:27.606466+0000","last_became_peered":"2024-09-15T23:36:27.606466+0000","last_unstale":"2024-09-15T23:36:28.621863+0000","last_undegraded":"2024-09-15T23:36:28.621863+0000","last_fullsized":"2024-09-15T23:36:28.621863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:55.070642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023954+0000","last_change":"2024-09-15T23:36:23.596605+0000","last_active":"2024-09-15T23:36:30.023954+0000","last_peered":"2024-09-15T23:36:30.023954+0000","last_clean":"2024-09-15T23:36:30.023954+0000","last_became_active":"2024-09-15T23:36:23.596318+0000","last_became_peered":"2024-09-15T23:36:23.596318+0000","last_unstale":"2024-09-15T23:36:30.023954+0000","last_undegraded":"2024-09-15T23:36:30.023954+0000","last_fullsized":"2024-09-15T23:36:30.023954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:02.746761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622853+0000","last_change":"2024-09-15T23:36:22.559703+0000","last_active":"2024-09-15T23:36:28.622853+0000","last_peered":"2024-09-15T23:36:28.622853+0000","last_clean":"2024-09-15T23:36:28.622853+0000","last_became_active":"2024-09-15T23:36:22.559271+0000","last_became_peered":"2024-09-15T23:36:22.559271+0000","last_unstale":"2024-09-15T23:36:28.622853+0000","last_undegraded":"2024-09-15T23:36:28.622853+0000","last_fullsized":"2024-09-15T23:36:28.622853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:23.347088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622571+0000","last_change":"2024-09-15T23:36:27.598188+0000","last_active":"2024-09-15T23:36:28.622571+0000","last_peered":"2024-09-15T23:36:28.622571+0000","last_clean":"2024-09-15T23:36:28.622571+0000","last_became_active":"2024-09-15T23:36:27.596219+0000","last_became_peered":"2024-09-15T23:36:27.596219+0000","last_unstale":"2024-09-15T23:36:28.622571+0000","last_undegraded":"2024-09-15T23:36:28.622571+0000","last_fullsized":"2024-09-15T23:36:28.622571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:23.776974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023619+0000","last_change":"2024-09-15T23:36:23.582672+0000","last_active":"2024-09-15T23:36:30.023619+0000","last_peered":"2024-09-15T23:36:30.023619+0000","last_clean":"2024-09-15T23:36:30.023619+0000","last_became_active":"2024-09-15T23:36:23.582388+0000","last_became_peered":"2024-09-15T23:36:23.582388+0000","last_unstale":"2024-09-15T23:36:30.023619+0000","last_undegraded":"2024-09-15T23:36:30.023619+0000","last_fullsized":"2024-09-15T23:36:30.023619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:07.005224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454431+0000","last_change":"2024-09-15T23:36:22.561458+0000","last_active":"2024-09-15T23:36:29.454431+0000","last_peered":"2024-09-15T23:36:29.454431+0000","last_clean":"2024-09-15T23:36:29.454431+0000","last_became_active":"2024-09-15T23:36:22.561171+0000","last_became_peered":"2024-09-15T23:36:22.561171+0000","last_unstale":"2024-09-15T23:36:29.454431+0000","last_undegraded":"2024-09-15T23:36:29.454431+0000","last_fullsized":"2024-09-15T23:36:29.454431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622564+0000","last_change":"2024-09-15T23:36:27.597617+0000","last_active":"2024-09-15T23:36:28.622564+0000","last_peered":"2024-09-15T23:36:28.622564+0000","last_clean":"2024-09-15T23:36:28.622564+0000","last_became_active":"2024-09-15T23:36:27.597224+0000","last_became_peered":"2024-09-15T23:36:27.597224+0000","last_unstale":"2024-09-15T23:36:28.622564+0000","last_undegraded":"2024-09-15T23:36:28.622564+0000","last_fullsized":"2024-09-15T23:36:28.622564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:06.783968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621714+0000","last_change":"2024-09-15T23:36:23.590567+0000","last_active":"2024-09-15T23:36:28.621714+0000","last_peered":"2024-09-15T23:36:28.621714+0000","last_clean":"2024-09-15T23:36:28.621714+0000","last_became_active":"2024-09-15T23:36:23.590272+0000","last_became_peered":"2024-09-15T23:36:23.590272+0000","last_unstale":"2024-09-15T23:36:28.621714+0000","last_undegraded":"2024-09-15T23:36:28.621714+0000","last_fullsized":"2024-09-15T23:36:28.621714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622169+0000","last_change":"2024-09-15T23:36:22.560601+0000","last_active":"2024-09-15T23:36:28.622169+0000","last_peered":"2024-09-15T23:36:28.622169+0000","last_clean":"2024-09-15T23:36:28.622169+0000","last_became_active":"2024-09-15T23:36:22.560306+0000","last_became_peered":"2024-09-15T23:36:22.560306+0000","last_unstale":"2024-09-15T23:36:28.622169+0000","last_undegraded":"2024-09-15T23:36:28.622169+0000","last_fullsized":"2024-09-15T23:36:28.622169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:38.283320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024190+0000","last_change":"2024-09-15T23:36:27.610042+0000","last_active":"2024-09-15T23:36:30.024190+0000","last_peered":"2024-09-15T23:36:30.024190+0000","last_clean":"2024-09-15T23:36:30.024190+0000","last_became_active":"2024-09-15T23:36:27.609630+0000","last_became_peered":"2024-09-15T23:36:27.609630+0000","last_unstale":"2024-09-15T23:36:30.024190+0000","last_undegraded":"2024-09-15T23:36:30.024190+0000","last_fullsized":"2024-09-15T23:36:30.024190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:03.827160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624171+0000","last_change":"2024-09-15T23:36:23.580635+0000","last_active":"2024-09-15T23:36:28.624171+0000","last_peered":"2024-09-15T23:36:28.624171+0000","last_clean":"2024-09-15T23:36:28.624171+0000","last_became_active":"2024-09-15T23:36:23.580348+0000","last_became_peered":"2024-09-15T23:36:23.580348+0000","last_unstale":"2024-09-15T23:36:28.624171+0000","last_undegraded":"2024-09-15T23:36:28.624171+0000","last_fullsized":"2024-09-15T23:36:28.624171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:03.646324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622420+0000","last_change":"2024-09-15T23:36:22.552162+0000","last_active":"2024-09-15T23:36:28.622420+0000","last_peered":"2024-09-15T23:36:28.622420+0000","last_clean":"2024-09-15T23:36:28.622420+0000","last_became_active":"2024-09-15T23:36:22.551788+0000","last_became_peered":"2024-09-15T23:36:22.551788+0000","last_unstale":"2024-09-15T23:36:28.622420+0000","last_undegraded":"2024-09-15T23:36:28.622420+0000","last_fullsized":"2024-09-15T23:36:28.622420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:04.332504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023778+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:36:30.023778+0000","last_peered":"2024-09-15T23:36:30.023778+0000","last_clean":"2024-09-15T23:36:30.023778+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:36:30.023778+0000","last_undegraded":"2024-09-15T23:36:30.023778+0000","last_fullsized":"2024-09-15T23:36:30.023778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623010+0000","last_change":"2024-09-15T23:36:27.604371+0000","last_active":"2024-09-15T23:36:28.623010+0000","last_peered":"2024-09-15T23:36:28.623010+0000","last_clean":"2024-09-15T23:36:28.623010+0000","last_became_active":"2024-09-15T23:36:27.603926+0000","last_became_peered":"2024-09-15T23:36:27.603926+0000","last_unstale":"2024-09-15T23:36:28.623010+0000","last_undegraded":"2024-09-15T23:36:28.623010+0000","last_fullsized":"2024-09-15T23:36:28.623010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:41.910981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622897+0000","last_change":"2024-09-15T23:36:23.575297+0000","last_active":"2024-09-15T23:36:28.622897+0000","last_peered":"2024-09-15T23:36:28.622897+0000","last_clean":"2024-09-15T23:36:28.622897+0000","last_became_active":"2024-09-15T23:36:23.574997+0000","last_became_peered":"2024-09-15T23:36:23.574997+0000","last_unstale":"2024-09-15T23:36:28.622897+0000","last_undegraded":"2024-09-15T23:36:28.622897+0000","last_fullsized":"2024-09-15T23:36:28.622897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:24.495389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622829+0000","last_change":"2024-09-15T23:36:22.569366+0000","last_active":"2024-09-15T23:36:28.622829+0000","last_peered":"2024-09-15T23:36:28.622829+0000","last_clean":"2024-09-15T23:36:28.622829+0000","last_became_active":"2024-09-15T23:36:22.569043+0000","last_became_peered":"2024-09-15T23:36:22.569043+0000","last_unstale":"2024-09-15T23:36:28.622829+0000","last_undegraded":"2024-09-15T23:36:28.622829+0000","last_fullsized":"2024-09-15T23:36:28.622829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:55.907462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024315+0000","last_change":"2024-09-15T23:36:27.611141+0000","last_active":"2024-09-15T23:36:30.024315+0000","last_peered":"2024-09-15T23:36:30.024315+0000","last_clean":"2024-09-15T23:36:30.024315+0000","last_became_active":"2024-09-15T23:36:27.610654+0000","last_became_peered":"2024-09-15T23:36:27.610654+0000","last_unstale":"2024-09-15T23:36:30.024315+0000","last_undegraded":"2024-09-15T23:36:30.024315+0000","last_fullsized":"2024-09-15T23:36:30.024315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:03.241943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025277+0000","last_change":"2024-09-15T23:36:23.592044+0000","last_active":"2024-09-15T23:36:30.025277+0000","last_peered":"2024-09-15T23:36:30.025277+0000","last_clean":"2024-09-15T23:36:30.025277+0000","last_became_active":"2024-09-15T23:36:23.591419+0000","last_became_peered":"2024-09-15T23:36:23.591419+0000","last_unstale":"2024-09-15T23:36:30.025277+0000","last_undegraded":"2024-09-15T23:36:30.025277+0000","last_fullsized":"2024-09-15T23:36:30.025277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:59.204736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024210+0000","last_change":"2024-09-15T23:36:22.568129+0000","last_active":"2024-09-15T23:36:30.024210+0000","last_peered":"2024-09-15T23:36:30.024210+0000","last_clean":"2024-09-15T23:36:30.024210+0000","last_became_active":"2024-09-15T23:36:22.567909+0000","last_became_peered":"2024-09-15T23:36:22.567909+0000","last_unstale":"2024-09-15T23:36:30.024210+0000","last_undegraded":"2024-09-15T23:36:30.024210+0000","last_fullsized":"2024-09-15T23:36:30.024210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:46.065174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024320+0000","last_change":"2024-09-15T23:36:27.610796+0000","last_active":"2024-09-15T23:36:30.024320+0000","last_peered":"2024-09-15T23:36:30.024320+0000","last_clean":"2024-09-15T23:36:30.024320+0000","last_became_active":"2024-09-15T23:36:27.610355+0000","last_became_peered":"2024-09-15T23:36:27.610355+0000","last_unstale":"2024-09-15T23:36:30.024320+0000","last_undegraded":"2024-09-15T23:36:30.024320+0000","last_fullsized":"2024-09-15T23:36:30.024320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:22.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024449+0000","last_change":"2024-09-15T23:36:23.592411+0000","last_active":"2024-09-15T23:36:30.024449+0000","last_peered":"2024-09-15T23:36:30.024449+0000","last_clean":"2024-09-15T23:36:30.024449+0000","last_became_active":"2024-09-15T23:36:23.592130+0000","last_became_peered":"2024-09-15T23:36:23.592130+0000","last_unstale":"2024-09-15T23:36:30.024449+0000","last_undegraded":"2024-09-15T23:36:30.024449+0000","last_fullsized":"2024-09-15T23:36:30.024449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.872038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623974+0000","last_change":"2024-09-15T23:36:22.560112+0000","last_active":"2024-09-15T23:36:28.623974+0000","last_peered":"2024-09-15T23:36:28.623974+0000","last_clean":"2024-09-15T23:36:28.623974+0000","last_became_active":"2024-09-15T23:36:22.559811+0000","last_became_peered":"2024-09-15T23:36:22.559811+0000","last_unstale":"2024-09-15T23:36:28.623974+0000","last_undegraded":"2024-09-15T23:36:28.623974+0000","last_fullsized":"2024-09-15T23:36:28.623974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:46.482072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621872+0000","last_change":"2024-09-15T23:36:27.603263+0000","last_active":"2024-09-15T23:36:28.621872+0000","last_peered":"2024-09-15T23:36:28.621872+0000","last_clean":"2024-09-15T23:36:28.621872+0000","last_became_active":"2024-09-15T23:36:27.602856+0000","last_became_peered":"2024-09-15T23:36:27.602856+0000","last_unstale":"2024-09-15T23:36:28.621872+0000","last_undegraded":"2024-09-15T23:36:28.621872+0000","last_fullsized":"2024-09-15T23:36:28.621872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:38.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024861+0000","last_change":"2024-09-15T23:36:23.594876+0000","last_active":"2024-09-15T23:36:30.024861+0000","last_peered":"2024-09-15T23:36:30.024861+0000","last_clean":"2024-09-15T23:36:30.024861+0000","last_became_active":"2024-09-15T23:36:23.594592+0000","last_became_peered":"2024-09-15T23:36:23.594592+0000","last_unstale":"2024-09-15T23:36:30.024861+0000","last_undegraded":"2024-09-15T23:36:30.024861+0000","last_fullsized":"2024-09-15T23:36:30.024861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:05.334429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024659+0000","last_change":"2024-09-15T23:36:22.568643+0000","last_active":"2024-09-15T23:36:30.024659+0000","last_peered":"2024-09-15T23:36:30.024659+0000","last_clean":"2024-09-15T23:36:30.024659+0000","last_became_active":"2024-09-15T23:36:22.568351+0000","last_became_peered":"2024-09-15T23:36:22.568351+0000","last_unstale":"2024-09-15T23:36:30.024659+0000","last_undegraded":"2024-09-15T23:36:30.024659+0000","last_fullsized":"2024-09-15T23:36:30.024659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:39.192475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023472+0000","last_change":"2024-09-15T23:36:27.605261+0000","last_active":"2024-09-15T23:36:30.023472+0000","last_peered":"2024-09-15T23:36:30.023472+0000","last_clean":"2024-09-15T23:36:30.023472+0000","last_became_active":"2024-09-15T23:36:27.604780+0000","last_became_peered":"2024-09-15T23:36:27.604780+0000","last_unstale":"2024-09-15T23:36:30.023472+0000","last_undegraded":"2024-09-15T23:36:30.023472+0000","last_fullsized":"2024-09-15T23:36:30.023472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:09.371636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621977+0000","last_change":"2024-09-15T23:36:23.582678+0000","last_active":"2024-09-15T23:36:28.621977+0000","last_peered":"2024-09-15T23:36:28.621977+0000","last_clean":"2024-09-15T23:36:28.621977+0000","last_became_active":"2024-09-15T23:36:23.582022+0000","last_became_peered":"2024-09-15T23:36:23.582022+0000","last_unstale":"2024-09-15T23:36:28.621977+0000","last_undegraded":"2024-09-15T23:36:28.621977+0000","last_fullsized":"2024-09-15T23:36:28.621977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:14.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622315+0000","last_change":"2024-09-15T23:36:22.552625+0000","last_active":"2024-09-15T23:36:28.622315+0000","last_peered":"2024-09-15T23:36:28.622315+0000","last_clean":"2024-09-15T23:36:28.622315+0000","last_became_active":"2024-09-15T23:36:22.552330+0000","last_became_peered":"2024-09-15T23:36:22.552330+0000","last_unstale":"2024-09-15T23:36:28.622315+0000","last_undegraded":"2024-09-15T23:36:28.622315+0000","last_fullsized":"2024-09-15T23:36:28.622315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:06:31.226510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024467+0000","last_change":"2024-09-15T23:36:27.607752+0000","last_active":"2024-09-15T23:36:30.024467+0000","last_peered":"2024-09-15T23:36:30.024467+0000","last_clean":"2024-09-15T23:36:30.024467+0000","last_became_active":"2024-09-15T23:36:27.607269+0000","last_became_peered":"2024-09-15T23:36:27.607269+0000","last_unstale":"2024-09-15T23:36:30.024467+0000","last_undegraded":"2024-09-15T23:36:30.024467+0000","last_fullsized":"2024-09-15T23:36:30.024467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:55.005028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023687+0000","last_change":"2024-09-15T23:36:23.585722+0000","last_active":"2024-09-15T23:36:30.023687+0000","last_peered":"2024-09-15T23:36:30.023687+0000","last_clean":"2024-09-15T23:36:30.023687+0000","last_became_active":"2024-09-15T23:36:23.585435+0000","last_became_peered":"2024-09-15T23:36:23.585435+0000","last_unstale":"2024-09-15T23:36:30.023687+0000","last_undegraded":"2024-09-15T23:36:30.023687+0000","last_fullsized":"2024-09-15T23:36:30.023687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023646+0000","last_change":"2024-09-15T23:36:22.568106+0000","last_active":"2024-09-15T23:36:30.023646+0000","last_peered":"2024-09-15T23:36:30.023646+0000","last_clean":"2024-09-15T23:36:30.023646+0000","last_became_active":"2024-09-15T23:36:22.567770+0000","last_became_peered":"2024-09-15T23:36:22.567770+0000","last_unstale":"2024-09-15T23:36:30.023646+0000","last_undegraded":"2024-09-15T23:36:30.023646+0000","last_fullsized":"2024-09-15T23:36:30.023646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:57.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024431+0000","last_change":"2024-09-15T23:36:27.611234+0000","last_active":"2024-09-15T23:36:30.024431+0000","last_peered":"2024-09-15T23:36:30.024431+0000","last_clean":"2024-09-15T23:36:30.024431+0000","last_became_active":"2024-09-15T23:36:27.610828+0000","last_became_peered":"2024-09-15T23:36:27.610828+0000","last_unstale":"2024-09-15T23:36:30.024431+0000","last_undegraded":"2024-09-15T23:36:30.024431+0000","last_fullsized":"2024-09-15T23:36:30.024431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:30.372516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622286+0000","last_change":"2024-09-15T23:36:23.592816+0000","last_active":"2024-09-15T23:36:28.622286+0000","last_peered":"2024-09-15T23:36:28.622286+0000","last_clean":"2024-09-15T23:36:28.622286+0000","last_became_active":"2024-09-15T23:36:23.590219+0000","last_became_peered":"2024-09-15T23:36:23.590219+0000","last_unstale":"2024-09-15T23:36:28.622286+0000","last_undegraded":"2024-09-15T23:36:28.622286+0000","last_fullsized":"2024-09-15T23:36:28.622286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:09:38.065436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622572+0000","last_change":"2024-09-15T23:36:22.555612+0000","last_active":"2024-09-15T23:36:28.622572+0000","last_peered":"2024-09-15T23:36:28.622572+0000","last_clean":"2024-09-15T23:36:28.622572+0000","last_became_active":"2024-09-15T23:36:22.555279+0000","last_became_peered":"2024-09-15T23:36:22.555279+0000","last_unstale":"2024-09-15T23:36:28.622572+0000","last_undegraded":"2024-09-15T23:36:28.622572+0000","last_fullsized":"2024-09-15T23:36:28.622572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:18.004944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622407+0000","last_change":"2024-09-15T23:36:27.594766+0000","last_active":"2024-09-15T23:36:28.622407+0000","last_peered":"2024-09-15T23:36:28.622407+0000","last_clean":"2024-09-15T23:36:28.622407+0000","last_became_active":"2024-09-15T23:36:27.594179+0000","last_became_peered":"2024-09-15T23:36:27.594179+0000","last_unstale":"2024-09-15T23:36:28.622407+0000","last_undegraded":"2024-09-15T23:36:28.622407+0000","last_fullsized":"2024-09-15T23:36:28.622407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:43.647541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622052+0000","last_change":"2024-09-15T23:36:23.595059+0000","last_active":"2024-09-15T23:36:28.622052+0000","last_peered":"2024-09-15T23:36:28.622052+0000","last_clean":"2024-09-15T23:36:28.622052+0000","last_became_active":"2024-09-15T23:36:23.594760+0000","last_became_peered":"2024-09-15T23:36:23.594760+0000","last_unstale":"2024-09-15T23:36:28.622052+0000","last_undegraded":"2024-09-15T23:36:28.622052+0000","last_fullsized":"2024-09-15T23:36:28.622052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:20.169043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454742+0000","last_change":"2024-09-15T23:36:22.569556+0000","last_active":"2024-09-15T23:36:29.454742+0000","last_peered":"2024-09-15T23:36:29.454742+0000","last_clean":"2024-09-15T23:36:29.454742+0000","last_became_active":"2024-09-15T23:36:22.569214+0000","last_became_peered":"2024-09-15T23:36:22.569214+0000","last_unstale":"2024-09-15T23:36:29.454742+0000","last_undegraded":"2024-09-15T23:36:29.454742+0000","last_fullsized":"2024-09-15T23:36:29.454742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:14:47.610942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622749+0000","last_change":"2024-09-15T23:36:27.601370+0000","last_active":"2024-09-15T23:36:28.622749+0000","last_peered":"2024-09-15T23:36:28.622749+0000","last_clean":"2024-09-15T23:36:28.622749+0000","last_became_active":"2024-09-15T23:36:27.600995+0000","last_became_peered":"2024-09-15T23:36:27.600995+0000","last_unstale":"2024-09-15T23:36:28.622749+0000","last_undegraded":"2024-09-15T23:36:28.622749+0000","last_fullsized":"2024-09-15T23:36:28.622749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:13.579126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025128+0000","last_change":"2024-09-15T23:36:23.595877+0000","last_active":"2024-09-15T23:36:30.025128+0000","last_peered":"2024-09-15T23:36:30.025128+0000","last_clean":"2024-09-15T23:36:30.025128+0000","last_became_active":"2024-09-15T23:36:23.595086+0000","last_became_peered":"2024-09-15T23:36:23.595086+0000","last_unstale":"2024-09-15T23:36:30.025128+0000","last_undegraded":"2024-09-15T23:36:30.025128+0000","last_fullsized":"2024-09-15T23:36:30.025128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:19.739767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:33.124757+0000","last_change":"2024-09-15T23:36:22.551636+0000","last_active":"2024-09-15T23:36:33.124757+0000","last_peered":"2024-09-15T23:36:33.124757+0000","last_clean":"2024-09-15T23:36:33.124757+0000","last_became_active":"2024-09-15T23:36:22.551316+0000","last_became_peered":"2024-09-15T23:36:22.551316+0000","last_unstale":"2024-09-15T23:36:33.124757+0000","last_undegraded":"2024-09-15T23:36:33.124757+0000","last_fullsized":"2024-09-15T23:36:33.124757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624158+0000","last_change":"2024-09-15T23:36:27.606629+0000","last_active":"2024-09-15T23:36:28.624158+0000","last_peered":"2024-09-15T23:36:28.624158+0000","last_clean":"2024-09-15T23:36:28.624158+0000","last_became_active":"2024-09-15T23:36:27.606224+0000","last_became_peered":"2024-09-15T23:36:27.606224+0000","last_unstale":"2024-09-15T23:36:28.624158+0000","last_undegraded":"2024-09-15T23:36:28.624158+0000","last_fullsized":"2024-09-15T23:36:28.624158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:09.576731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622435+0000","last_change":"2024-09-15T23:36:23.580904+0000","last_active":"2024-09-15T23:36:28.622435+0000","last_peered":"2024-09-15T23:36:28.622435+0000","last_clean":"2024-09-15T23:36:28.622435+0000","last_became_active":"2024-09-15T23:36:23.580459+0000","last_became_peered":"2024-09-15T23:36:23.580459+0000","last_unstale":"2024-09-15T23:36:28.622435+0000","last_undegraded":"2024-09-15T23:36:28.622435+0000","last_fullsized":"2024-09-15T23:36:28.622435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:50.078320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454752+0000","last_change":"2024-09-15T23:36:22.557264+0000","last_active":"2024-09-15T23:36:29.454752+0000","last_peered":"2024-09-15T23:36:29.454752+0000","last_clean":"2024-09-15T23:36:29.454752+0000","last_became_active":"2024-09-15T23:36:22.556929+0000","last_became_peered":"2024-09-15T23:36:22.556929+0000","last_unstale":"2024-09-15T23:36:29.454752+0000","last_undegraded":"2024-09-15T23:36:29.454752+0000","last_fullsized":"2024-09-15T23:36:29.454752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:57.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024486+0000","last_change":"2024-09-15T23:36:27.605955+0000","last_active":"2024-09-15T23:36:30.024486+0000","last_peered":"2024-09-15T23:36:30.024486+0000","last_clean":"2024-09-15T23:36:30.024486+0000","last_became_active":"2024-09-15T23:36:27.605521+0000","last_became_peered":"2024-09-15T23:36:27.605521+0000","last_unstale":"2024-09-15T23:36:30.024486+0000","last_undegraded":"2024-09-15T23:36:30.024486+0000","last_fullsized":"2024-09-15T23:36:30.024486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:23.776974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024511+0000","last_change":"2024-09-15T23:36:23.583781+0000","last_active":"2024-09-15T23:36:30.024511+0000","last_peered":"2024-09-15T23:36:30.024511+0000","last_clean":"2024-09-15T23:36:30.024511+0000","last_became_active":"2024-09-15T23:36:23.583475+0000","last_became_peered":"2024-09-15T23:36:23.583475+0000","last_unstale":"2024-09-15T23:36:30.024511+0000","last_undegraded":"2024-09-15T23:36:30.024511+0000","last_fullsized":"2024-09-15T23:36:30.024511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:23.983256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024984+0000","last_change":"2024-09-15T23:36:22.569943+0000","last_active":"2024-09-15T23:36:30.024984+0000","last_peered":"2024-09-15T23:36:30.024984+0000","last_clean":"2024-09-15T23:36:30.024984+0000","last_became_active":"2024-09-15T23:36:22.568617+0000","last_became_peered":"2024-09-15T23:36:22.568617+0000","last_unstale":"2024-09-15T23:36:30.024984+0000","last_undegraded":"2024-09-15T23:36:30.024984+0000","last_fullsized":"2024-09-15T23:36:30.024984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:44.229130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454281+0000","last_change":"2024-09-15T23:36:27.604270+0000","last_active":"2024-09-15T23:36:29.454281+0000","last_peered":"2024-09-15T23:36:29.454281+0000","last_clean":"2024-09-15T23:36:29.454281+0000","last_became_active":"2024-09-15T23:36:27.603416+0000","last_became_peered":"2024-09-15T23:36:27.603416+0000","last_unstale":"2024-09-15T23:36:29.454281+0000","last_undegraded":"2024-09-15T23:36:29.454281+0000","last_fullsized":"2024-09-15T23:36:29.454281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:12.747026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024835+0000","last_change":"2024-09-15T23:36:23.580585+0000","last_active":"2024-09-15T23:36:30.024835+0000","last_peered":"2024-09-15T23:36:30.024835+0000","last_clean":"2024-09-15T23:36:30.024835+0000","last_became_active":"2024-09-15T23:36:23.579003+0000","last_became_peered":"2024-09-15T23:36:23.579003+0000","last_unstale":"2024-09-15T23:36:30.024835+0000","last_undegraded":"2024-09-15T23:36:30.024835+0000","last_fullsized":"2024-09-15T23:36:30.024835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:24.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025091+0000","last_change":"2024-09-15T23:36:22.557388+0000","last_active":"2024-09-15T23:36:30.025091+0000","last_peered":"2024-09-15T23:36:30.025091+0000","last_clean":"2024-09-15T23:36:30.025091+0000","last_became_active":"2024-09-15T23:36:22.556138+0000","last_became_peered":"2024-09-15T23:36:22.556138+0000","last_unstale":"2024-09-15T23:36:30.025091+0000","last_undegraded":"2024-09-15T23:36:30.025091+0000","last_fullsized":"2024-09-15T23:36:30.025091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623557+0000","last_change":"2024-09-15T23:36:27.600449+0000","last_active":"2024-09-15T23:36:28.623557+0000","last_peered":"2024-09-15T23:36:28.623557+0000","last_clean":"2024-09-15T23:36:28.623557+0000","last_became_active":"2024-09-15T23:36:27.599937+0000","last_became_peered":"2024-09-15T23:36:27.599937+0000","last_unstale":"2024-09-15T23:36:28.623557+0000","last_undegraded":"2024-09-15T23:36:28.623557+0000","last_fullsized":"2024-09-15T23:36:28.623557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:13.541381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623785+0000","last_change":"2024-09-15T23:36:27.607667+0000","last_active":"2024-09-15T23:36:28.623785+0000","last_peered":"2024-09-15T23:36:28.623785+0000","last_clean":"2024-09-15T23:36:28.623785+0000","last_became_active":"2024-09-15T23:36:27.607227+0000","last_became_peered":"2024-09-15T23:36:27.607227+0000","last_unstale":"2024-09-15T23:36:28.623785+0000","last_undegraded":"2024-09-15T23:36:28.623785+0000","last_fullsized":"2024-09-15T23:36:28.623785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:33.572386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623619+0000","last_change":"2024-09-15T23:36:23.579014+0000","last_active":"2024-09-15T23:36:28.623619+0000","last_peered":"2024-09-15T23:36:28.623619+0000","last_clean":"2024-09-15T23:36:28.623619+0000","last_became_active":"2024-09-15T23:36:23.578709+0000","last_became_peered":"2024-09-15T23:36:23.578709+0000","last_unstale":"2024-09-15T23:36:28.623619+0000","last_undegraded":"2024-09-15T23:36:28.623619+0000","last_fullsized":"2024-09-15T23:36:28.623619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:07.005224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622050+0000","last_change":"2024-09-15T23:36:22.556649+0000","last_active":"2024-09-15T23:36:28.622050+0000","last_peered":"2024-09-15T23:36:28.622050+0000","last_clean":"2024-09-15T23:36:28.622050+0000","last_became_active":"2024-09-15T23:36:22.556332+0000","last_became_peered":"2024-09-15T23:36:22.556332+0000","last_unstale":"2024-09-15T23:36:28.622050+0000","last_undegraded":"2024-09-15T23:36:28.622050+0000","last_fullsized":"2024-09-15T23:36:28.622050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:46.855766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623003+0000","last_change":"2024-09-15T23:36:27.612447+0000","last_active":"2024-09-15T23:36:28.623003+0000","last_peered":"2024-09-15T23:36:28.623003+0000","last_clean":"2024-09-15T23:36:28.623003+0000","last_became_active":"2024-09-15T23:36:27.612007+0000","last_became_peered":"2024-09-15T23:36:27.612007+0000","last_unstale":"2024-09-15T23:36:28.623003+0000","last_undegraded":"2024-09-15T23:36:28.623003+0000","last_fullsized":"2024-09-15T23:36:28.623003+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:15.233014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023818+0000","last_change":"2024-09-15T23:36:23.592928+0000","last_active":"2024-09-15T23:36:30.023818+0000","last_peered":"2024-09-15T23:36:30.023818+0000","last_clean":"2024-09-15T23:36:30.023818+0000","last_became_active":"2024-09-15T23:36:23.592641+0000","last_became_peered":"2024-09-15T23:36:23.592641+0000","last_unstale":"2024-09-15T23:36:30.023818+0000","last_undegraded":"2024-09-15T23:36:30.023818+0000","last_fullsized":"2024-09-15T23:36:30.023818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:02.943531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622171+0000","last_change":"2024-09-15T23:36:22.569415+0000","last_active":"2024-09-15T23:36:28.622171+0000","last_peered":"2024-09-15T23:36:28.622171+0000","last_clean":"2024-09-15T23:36:28.622171+0000","last_became_active":"2024-09-15T23:36:22.569105+0000","last_became_peered":"2024-09-15T23:36:22.569105+0000","last_unstale":"2024-09-15T23:36:28.622171+0000","last_undegraded":"2024-09-15T23:36:28.622171+0000","last_fullsized":"2024-09-15T23:36:28.622171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622420+0000","last_change":"2024-09-15T23:36:22.569714+0000","last_active":"2024-09-15T23:36:28.622420+0000","last_peered":"2024-09-15T23:36:28.622420+0000","last_clean":"2024-09-15T23:36:28.622420+0000","last_became_active":"2024-09-15T23:36:22.569371+0000","last_became_peered":"2024-09-15T23:36:22.569371+0000","last_unstale":"2024-09-15T23:36:28.622420+0000","last_undegraded":"2024-09-15T23:36:28.622420+0000","last_fullsized":"2024-09-15T23:36:28.622420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:19.167117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622958+0000","last_change":"2024-09-15T23:36:23.591060+0000","last_active":"2024-09-15T23:36:28.622958+0000","last_peered":"2024-09-15T23:36:28.622958+0000","last_clean":"2024-09-15T23:36:28.622958+0000","last_became_active":"2024-09-15T23:36:23.590748+0000","last_became_peered":"2024-09-15T23:36:23.590748+0000","last_unstale":"2024-09-15T23:36:28.622958+0000","last_undegraded":"2024-09-15T23:36:28.622958+0000","last_fullsized":"2024-09-15T23:36:28.622958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:56.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624452+0000","last_change":"2024-09-15T23:36:27.602410+0000","last_active":"2024-09-15T23:36:28.624452+0000","last_peered":"2024-09-15T23:36:28.624452+0000","last_clean":"2024-09-15T23:36:28.624452+0000","last_became_active":"2024-09-15T23:36:27.602009+0000","last_became_peered":"2024-09-15T23:36:27.602009+0000","last_unstale":"2024-09-15T23:36:28.624452+0000","last_undegraded":"2024-09-15T23:36:28.624452+0000","last_fullsized":"2024-09-15T23:36:28.624452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:38.882900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024165+0000","last_change":"2024-09-15T23:36:22.562409+0000","last_active":"2024-09-15T23:36:30.024165+0000","last_peered":"2024-09-15T23:36:30.024165+0000","last_clean":"2024-09-15T23:36:30.024165+0000","last_became_active":"2024-09-15T23:36:22.561579+0000","last_became_peered":"2024-09-15T23:36:22.561579+0000","last_unstale":"2024-09-15T23:36:30.024165+0000","last_undegraded":"2024-09-15T23:36:30.024165+0000","last_fullsized":"2024-09-15T23:36:30.024165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:46.879331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023841+0000","last_change":"2024-09-15T23:36:23.584956+0000","last_active":"2024-09-15T23:36:30.023841+0000","last_peered":"2024-09-15T23:36:30.023841+0000","last_clean":"2024-09-15T23:36:30.023841+0000","last_became_active":"2024-09-15T23:36:23.584680+0000","last_became_peered":"2024-09-15T23:36:23.584680+0000","last_unstale":"2024-09-15T23:36:30.023841+0000","last_undegraded":"2024-09-15T23:36:30.023841+0000","last_fullsized":"2024-09-15T23:36:30.023841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454641+0000","last_change":"2024-09-15T23:36:27.603930+0000","last_active":"2024-09-15T23:36:29.454641+0000","last_peered":"2024-09-15T23:36:29.454641+0000","last_clean":"2024-09-15T23:36:29.454641+0000","last_became_active":"2024-09-15T23:36:27.603486+0000","last_became_peered":"2024-09-15T23:36:27.603486+0000","last_unstale":"2024-09-15T23:36:29.454641+0000","last_undegraded":"2024-09-15T23:36:29.454641+0000","last_fullsized":"2024-09-15T23:36:29.454641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:41.146991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024615+0000","last_change":"2024-09-15T23:36:23.590531+0000","last_active":"2024-09-15T23:36:30.024615+0000","last_peered":"2024-09-15T23:36:30.024615+0000","last_clean":"2024-09-15T23:36:30.024615+0000","last_became_active":"2024-09-15T23:36:23.590278+0000","last_became_peered":"2024-09-15T23:36:23.590278+0000","last_unstale":"2024-09-15T23:36:30.024615+0000","last_undegraded":"2024-09-15T23:36:30.024615+0000","last_fullsized":"2024-09-15T23:36:30.024615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454276+0000","last_change":"2024-09-15T23:36:23.580382+0000","last_active":"2024-09-15T23:36:29.454276+0000","last_peered":"2024-09-15T23:36:29.454276+0000","last_clean":"2024-09-15T23:36:29.454276+0000","last_became_active":"2024-09-15T23:36:23.580096+0000","last_became_peered":"2024-09-15T23:36:23.580096+0000","last_unstale":"2024-09-15T23:36:29.454276+0000","last_undegraded":"2024-09-15T23:36:29.454276+0000","last_fullsized":"2024-09-15T23:36:29.454276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622200+0000","last_change":"2024-09-15T23:36:23.587488+0000","last_active":"2024-09-15T23:36:28.622200+0000","last_peered":"2024-09-15T23:36:28.622200+0000","last_clean":"2024-09-15T23:36:28.622200+0000","last_became_active":"2024-09-15T23:36:23.587199+0000","last_became_peered":"2024-09-15T23:36:23.587199+0000","last_unstale":"2024-09-15T23:36:28.622200+0000","last_undegraded":"2024-09-15T23:36:28.622200+0000","last_fullsized":"2024-09-15T23:36:28.622200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:41.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023999+0000","last_change":"2024-09-15T23:36:23.583971+0000","last_active":"2024-09-15T23:36:30.023999+0000","last_peered":"2024-09-15T23:36:30.023999+0000","last_clean":"2024-09-15T23:36:30.023999+0000","last_became_active":"2024-09-15T23:36:23.583670+0000","last_became_peered":"2024-09-15T23:36:23.583670+0000","last_unstale":"2024-09-15T23:36:30.023999+0000","last_undegraded":"2024-09-15T23:36:30.023999+0000","last_fullsized":"2024-09-15T23:36:30.023999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:56.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622689+0000","last_change":"2024-09-15T23:36:23.598604+0000","last_active":"2024-09-15T23:36:28.622689+0000","last_peered":"2024-09-15T23:36:28.622689+0000","last_clean":"2024-09-15T23:36:28.622689+0000","last_became_active":"2024-09-15T23:36:23.598283+0000","last_became_peered":"2024-09-15T23:36:23.598283+0000","last_unstale":"2024-09-15T23:36:28.622689+0000","last_undegraded":"2024-09-15T23:36:28.622689+0000","last_fullsized":"2024-09-15T23:36:28.622689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:46.930120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622563+0000","last_change":"2024-09-15T23:36:23.588957+0000","last_active":"2024-09-15T23:36:28.622563+0000","last_peered":"2024-09-15T23:36:28.622563+0000","last_clean":"2024-09-15T23:36:28.622563+0000","last_became_active":"2024-09-15T23:36:23.588651+0000","last_became_peered":"2024-09-15T23:36:23.588651+0000","last_unstale":"2024-09-15T23:36:28.622563+0000","last_undegraded":"2024-09-15T23:36:28.622563+0000","last_fullsized":"2024-09-15T23:36:28.622563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:05.334429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024765+0000","last_change":"2024-09-15T23:36:23.579681+0000","last_active":"2024-09-15T23:36:30.024765+0000","last_peered":"2024-09-15T23:36:30.024765+0000","last_clean":"2024-09-15T23:36:30.024765+0000","last_became_active":"2024-09-15T23:36:23.579299+0000","last_became_peered":"2024-09-15T23:36:23.579299+0000","last_unstale":"2024-09-15T23:36:30.024765+0000","last_undegraded":"2024-09-15T23:36:30.024765+0000","last_fullsized":"2024-09-15T23:36:30.024765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:19.006869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024346+0000","last_change":"2024-09-15T23:36:23.587478+0000","last_active":"2024-09-15T23:36:30.024346+0000","last_peered":"2024-09-15T23:36:30.024346+0000","last_clean":"2024-09-15T23:36:30.024346+0000","last_became_active":"2024-09-15T23:36:23.586993+0000","last_became_peered":"2024-09-15T23:36:23.586993+0000","last_unstale":"2024-09-15T23:36:30.024346+0000","last_undegraded":"2024-09-15T23:36:30.024346+0000","last_fullsized":"2024-09-15T23:36:30.024346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:11:50.246113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622283+0000","last_change":"2024-09-15T23:36:23.594827+0000","last_active":"2024-09-15T23:36:28.622283+0000","last_peered":"2024-09-15T23:36:28.622283+0000","last_clean":"2024-09-15T23:36:28.622283+0000","last_became_active":"2024-09-15T23:36:23.594542+0000","last_became_peered":"2024-09-15T23:36:23.594542+0000","last_unstale":"2024-09-15T23:36:28.622283+0000","last_undegraded":"2024-09-15T23:36:28.622283+0000","last_fullsized":"2024-09-15T23:36:28.622283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.872038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023772+0000","last_change":"2024-09-15T23:36:23.588492+0000","last_active":"2024-09-15T23:36:30.023772+0000","last_peered":"2024-09-15T23:36:30.023772+0000","last_clean":"2024-09-15T23:36:30.023772+0000","last_became_active":"2024-09-15T23:36:23.588154+0000","last_became_peered":"2024-09-15T23:36:23.588154+0000","last_unstale":"2024-09-15T23:36:30.023772+0000","last_undegraded":"2024-09-15T23:36:30.023772+0000","last_fullsized":"2024-09-15T23:36:30.023772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:50:24.116022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023752+0000","last_change":"2024-09-15T23:36:23.584149+0000","last_active":"2024-09-15T23:36:30.023752+0000","last_peered":"2024-09-15T23:36:30.023752+0000","last_clean":"2024-09-15T23:36:30.023752+0000","last_became_active":"2024-09-15T23:36:23.583864+0000","last_became_peered":"2024-09-15T23:36:23.583864+0000","last_unstale":"2024-09-15T23:36:30.023752+0000","last_undegraded":"2024-09-15T23:36:30.023752+0000","last_fullsized":"2024-09-15T23:36:30.023752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:09.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622823+0000","last_change":"2024-09-15T23:36:23.581142+0000","last_active":"2024-09-15T23:36:28.622823+0000","last_peered":"2024-09-15T23:36:28.622823+0000","last_clean":"2024-09-15T23:36:28.622823+0000","last_became_active":"2024-09-15T23:36:23.580551+0000","last_became_peered":"2024-09-15T23:36:23.580551+0000","last_unstale":"2024-09-15T23:36:28.622823+0000","last_undegraded":"2024-09-15T23:36:28.622823+0000","last_fullsized":"2024-09-15T23:36:28.622823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:44.358074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025231+0000","last_change":"2024-09-15T23:36:23.596339+0000","last_active":"2024-09-15T23:36:30.025231+0000","last_peered":"2024-09-15T23:36:30.025231+0000","last_clean":"2024-09-15T23:36:30.025231+0000","last_became_active":"2024-09-15T23:36:23.590862+0000","last_became_peered":"2024-09-15T23:36:23.590862+0000","last_unstale":"2024-09-15T23:36:30.025231+0000","last_undegraded":"2024-09-15T23:36:30.025231+0000","last_fullsized":"2024-09-15T23:36:30.025231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:06.543007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623049+0000","last_change":"2024-09-15T23:36:23.589509+0000","last_active":"2024-09-15T23:36:28.623049+0000","last_peered":"2024-09-15T23:36:28.623049+0000","last_clean":"2024-09-15T23:36:28.623049+0000","last_became_active":"2024-09-15T23:36:23.589201+0000","last_became_peered":"2024-09-15T23:36:23.589201+0000","last_unstale":"2024-09-15T23:36:28.623049+0000","last_undegraded":"2024-09-15T23:36:28.623049+0000","last_fullsized":"2024-09-15T23:36:28.623049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:02.514262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454968+0000","last_change":"2024-09-15T23:36:23.582505+0000","last_active":"2024-09-15T23:36:29.454968+0000","last_peered":"2024-09-15T23:36:29.454968+0000","last_clean":"2024-09-15T23:36:29.454968+0000","last_became_active":"2024-09-15T23:36:23.582210+0000","last_became_peered":"2024-09-15T23:36:23.582210+0000","last_unstale":"2024-09-15T23:36:29.454968+0000","last_undegraded":"2024-09-15T23:36:29.454968+0000","last_fullsized":"2024-09-15T23:36:29.454968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623156+0000","last_change":"2024-09-15T23:36:23.590697+0000","last_active":"2024-09-15T23:36:28.623156+0000","last_peered":"2024-09-15T23:36:28.623156+0000","last_clean":"2024-09-15T23:36:28.623156+0000","last_became_active":"2024-09-15T23:36:23.590399+0000","last_became_peered":"2024-09-15T23:36:23.590399+0000","last_unstale":"2024-09-15T23:36:28.623156+0000","last_undegraded":"2024-09-15T23:36:28.623156+0000","last_fullsized":"2024-09-15T23:36:28.623156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.804071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621897+0000","last_change":"2024-09-15T23:36:23.583813+0000","last_active":"2024-09-15T23:36:28.621897+0000","last_peered":"2024-09-15T23:36:28.621897+0000","last_clean":"2024-09-15T23:36:28.621897+0000","last_became_active":"2024-09-15T23:36:23.583516+0000","last_became_peered":"2024-09-15T23:36:23.583516+0000","last_unstale":"2024-09-15T23:36:28.621897+0000","last_undegraded":"2024-09-15T23:36:28.621897+0000","last_fullsized":"2024-09-15T23:36:28.621897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:40.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023592+0000","last_change":"2024-09-15T23:36:23.587846+0000","last_active":"2024-09-15T23:36:30.023592+0000","last_peered":"2024-09-15T23:36:30.023592+0000","last_clean":"2024-09-15T23:36:30.023592+0000","last_became_active":"2024-09-15T23:36:23.587540+0000","last_became_peered":"2024-09-15T23:36:23.587540+0000","last_unstale":"2024-09-15T23:36:30.023592+0000","last_undegraded":"2024-09-15T23:36:30.023592+0000","last_fullsized":"2024-09-15T23:36:30.023592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:31.212365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621934+0000","last_change":"2024-09-15T23:36:23.589402+0000","last_active":"2024-09-15T23:36:28.621934+0000","last_peered":"2024-09-15T23:36:28.621934+0000","last_clean":"2024-09-15T23:36:28.621934+0000","last_became_active":"2024-09-15T23:36:23.589096+0000","last_became_peered":"2024-09-15T23:36:23.589096+0000","last_unstale":"2024-09-15T23:36:28.621934+0000","last_undegraded":"2024-09-15T23:36:28.621934+0000","last_fullsized":"2024-09-15T23:36:28.621934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:19.006869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621955+0000","last_change":"2024-09-15T23:36:23.598740+0000","last_active":"2024-09-15T23:36:28.621955+0000","last_peered":"2024-09-15T23:36:28.621955+0000","last_clean":"2024-09-15T23:36:28.621955+0000","last_became_active":"2024-09-15T23:36:23.598307+0000","last_became_peered":"2024-09-15T23:36:23.598307+0000","last_unstale":"2024-09-15T23:36:28.621955+0000","last_undegraded":"2024-09-15T23:36:28.621955+0000","last_fullsized":"2024-09-15T23:36:28.621955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:24.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025141+0000","last_change":"2024-09-15T23:36:23.591784+0000","last_active":"2024-09-15T23:36:30.025141+0000","last_peered":"2024-09-15T23:36:30.025141+0000","last_clean":"2024-09-15T23:36:30.025141+0000","last_became_active":"2024-09-15T23:36:23.591022+0000","last_became_peered":"2024-09-15T23:36:23.591022+0000","last_unstale":"2024-09-15T23:36:30.025141+0000","last_undegraded":"2024-09-15T23:36:30.025141+0000","last_fullsized":"2024-09-15T23:36:30.025141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:23.323481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025097+0000","last_change":"2024-09-15T23:36:23.586049+0000","last_active":"2024-09-15T23:36:30.025097+0000","last_peered":"2024-09-15T23:36:30.025097+0000","last_clean":"2024-09-15T23:36:30.025097+0000","last_became_active":"2024-09-15T23:36:23.585679+0000","last_became_peered":"2024-09-15T23:36:23.585679+0000","last_unstale":"2024-09-15T23:36:30.025097+0000","last_undegraded":"2024-09-15T23:36:30.025097+0000","last_fullsized":"2024-09-15T23:36:30.025097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:39.610334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024825+0000","last_change":"2024-09-15T23:36:23.595842+0000","last_active":"2024-09-15T23:36:30.024825+0000","last_peered":"2024-09-15T23:36:30.024825+0000","last_clean":"2024-09-15T23:36:30.024825+0000","last_became_active":"2024-09-15T23:36:23.595526+0000","last_became_peered":"2024-09-15T23:36:23.595526+0000","last_unstale":"2024-09-15T23:36:30.024825+0000","last_undegraded":"2024-09-15T23:36:30.024825+0000","last_fullsized":"2024-09-15T23:36:30.024825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:37.109784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025005+0000","last_change":"2024-09-15T23:36:23.572643+0000","last_active":"2024-09-15T23:36:30.025005+0000","last_peered":"2024-09-15T23:36:30.025005+0000","last_clean":"2024-09-15T23:36:30.025005+0000","last_became_active":"2024-09-15T23:36:23.572340+0000","last_became_peered":"2024-09-15T23:36:23.572340+0000","last_unstale":"2024-09-15T23:36:30.025005+0000","last_undegraded":"2024-09-15T23:36:30.025005+0000","last_fullsized":"2024-09-15T23:36:30.025005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023623+0000","last_change":"2024-09-15T23:36:23.592206+0000","last_active":"2024-09-15T23:36:30.023623+0000","last_peered":"2024-09-15T23:36:30.023623+0000","last_clean":"2024-09-15T23:36:30.023623+0000","last_became_active":"2024-09-15T23:36:23.590955+0000","last_became_peered":"2024-09-15T23:36:23.590955+0000","last_unstale":"2024-09-15T23:36:30.023623+0000","last_undegraded":"2024-09-15T23:36:30.023623+0000","last_fullsized":"2024-09-15T23:36:30.023623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:17.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023548+0000","last_change":"2024-09-15T23:36:23.594637+0000","last_active":"2024-09-15T23:36:30.023548+0000","last_peered":"2024-09-15T23:36:30.023548+0000","last_clean":"2024-09-15T23:36:30.023548+0000","last_became_active":"2024-09-15T23:36:23.594333+0000","last_became_peered":"2024-09-15T23:36:23.594333+0000","last_unstale":"2024-09-15T23:36:30.023548+0000","last_undegraded":"2024-09-15T23:36:30.023548+0000","last_fullsized":"2024-09-15T23:36:30.023548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:51.033435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024411+0000","last_change":"2024-09-15T23:36:23.592276+0000","last_active":"2024-09-15T23:36:30.024411+0000","last_peered":"2024-09-15T23:36:30.024411+0000","last_clean":"2024-09-15T23:36:30.024411+0000","last_became_active":"2024-09-15T23:36:23.591988+0000","last_became_peered":"2024-09-15T23:36:23.591988+0000","last_unstale":"2024-09-15T23:36:30.024411+0000","last_undegraded":"2024-09-15T23:36:30.024411+0000","last_fullsized":"2024-09-15T23:36:30.024411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024046+0000","last_change":"2024-09-15T23:36:23.596925+0000","last_active":"2024-09-15T23:36:30.024046+0000","last_peered":"2024-09-15T23:36:30.024046+0000","last_clean":"2024-09-15T23:36:30.024046+0000","last_became_active":"2024-09-15T23:36:23.596610+0000","last_became_peered":"2024-09-15T23:36:23.596610+0000","last_unstale":"2024-09-15T23:36:30.024046+0000","last_undegraded":"2024-09-15T23:36:30.024046+0000","last_fullsized":"2024-09-15T23:36:30.024046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:56.256643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023789+0000","last_change":"2024-09-15T23:36:23.591422+0000","last_active":"2024-09-15T23:36:30.023789+0000","last_peered":"2024-09-15T23:36:30.023789+0000","last_clean":"2024-09-15T23:36:30.023789+0000","last_became_active":"2024-09-15T23:36:23.590380+0000","last_became_peered":"2024-09-15T23:36:23.590380+0000","last_unstale":"2024-09-15T23:36:30.023789+0000","last_undegraded":"2024-09-15T23:36:30.023789+0000","last_fullsized":"2024-09-15T23:36:30.023789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:20.169043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024050+0000","last_change":"2024-09-15T23:36:23.592150+0000","last_active":"2024-09-15T23:36:30.024050+0000","last_peered":"2024-09-15T23:36:30.024050+0000","last_clean":"2024-09-15T23:36:30.024050+0000","last_became_active":"2024-09-15T23:36:23.591205+0000","last_became_peered":"2024-09-15T23:36:23.591205+0000","last_unstale":"2024-09-15T23:36:30.024050+0000","last_undegraded":"2024-09-15T23:36:30.024050+0000","last_fullsized":"2024-09-15T23:36:30.024050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:50.950584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024247+0000","last_change":"2024-09-15T23:36:23.593241+0000","last_active":"2024-09-15T23:36:30.024247+0000","last_peered":"2024-09-15T23:36:30.024247+0000","last_clean":"2024-09-15T23:36:30.024247+0000","last_became_active":"2024-09-15T23:36:23.591733+0000","last_became_peered":"2024-09-15T23:36:23.591733+0000","last_unstale":"2024-09-15T23:36:30.024247+0000","last_undegraded":"2024-09-15T23:36:30.024247+0000","last_fullsized":"2024-09-15T23:36:30.024247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:23.692197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024357+0000","last_change":"2024-09-15T23:36:23.596045+0000","last_active":"2024-09-15T23:36:30.024357+0000","last_peered":"2024-09-15T23:36:30.024357+0000","last_clean":"2024-09-15T23:36:30.024357+0000","last_became_active":"2024-09-15T23:36:23.595078+0000","last_became_peered":"2024-09-15T23:36:23.595078+0000","last_unstale":"2024-09-15T23:36:30.024357+0000","last_undegraded":"2024-09-15T23:36:30.024357+0000","last_fullsized":"2024-09-15T23:36:30.024357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:40.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":262144,"data_stored":103393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":270336,"data_stored":106655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":856064,"data_stored":693249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":102867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":102935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":258048,"data_stored":102811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":860160,"data_stored":693271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":107327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:36:44.220 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T23:36:44.221 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T23:36:44.221 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T23:36:44.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T23:36:44.221 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T23:36:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.356+0000 7fb8f549b640 1 Processor -- start 2024-09-15T23:36:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.356+0000 7fb8f549b640 1 -- start start 2024-09-15T23:36:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8f549b640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb8f009b610 0x7fb8f00ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8f549b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f00d14a0 con 0x7fb8f00ce590 2024-09-15T23:36:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8f549b640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f00d1780 con 0x7fb8f015e870 2024-09-15T23:36:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8f549b640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb8f015ec50 con 0x7fb8f009b610 2024-09-15T23:36:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8ef7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8f00ce590 0x7fb8f006ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44640/0 (socket says 172.21.15.7:44640) 2024-09-15T23:36:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8ef7fe640 1 -- 172.21.15.7:0/712438518 learned_addr learned my addr 172.21.15.7:0/712438518 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8ee7fc640 1 --2- 172.21.15.7:0/712438518 >> v2:172.21.15.18:3300/0 conn(0x7fb8f009b610 0x7fb8f00ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8edffb640 1 -- 172.21.15.7:0/712438518 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035873798 0 0) 0x7fb8f00d14a0 con 0x7fb8f00ce590 2024-09-15T23:36:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8edffb640 1 -- 172.21.15.7:0/712438518 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8dc003520 con 0x7fb8f00ce590 2024-09-15T23:36:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8ee7fc640 1 -- 172.21.15.7:0/712438518 >> v1:172.21.15.18:6789/0 conn(0x7fb8f015e870 legacy=0x7fb8f00d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8ee7fc640 1 -- 172.21.15.7:0/712438518 >> v1:172.21.15.7:6789/0 conn(0x7fb8f00ce590 legacy=0x7fb8f006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.357+0000 7fb8ee7fc640 1 -- 172.21.15.7:0/712438518 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f00cd850 con 0x7fb8f009b610 2024-09-15T23:36:44.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.358+0000 7fb8ee7fc640 1 --2- 172.21.15.7:0/712438518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f009b610 0x7fb8f00ce050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80097a0 tx=0x7fb8d802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7cccba04d6b896 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:44.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.358+0000 7fb8edffb640 1 -- 172.21.15.7:0/712438518 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8037070 con 0x7fb8f009b610 2024-09-15T23:36:44.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.358+0000 7fb8edffb640 1 -- 172.21.15.7:0/712438518 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d8032640 con 0x7fb8f009b610 2024-09-15T23:36:44.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.358+0000 7fb8edffb640 1 -- 172.21.15.7:0/712438518 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8032a10 con 0x7fb8f009b610 2024-09-15T23:36:44.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.359+0000 7fb8f549b640 1 -- 172.21.15.7:0/712438518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f009b610 msgr2=0x7fb8f00ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.359+0000 7fb8f549b640 1 --2- 172.21.15.7:0/712438518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f009b610 0x7fb8f00ce050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80097a0 tx=0x7fb8d802a2c0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.359+0000 7fb8f549b640 1 -- 172.21.15.7:0/712438518 shutdown_connections 2024-09-15T23:36:44.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.359+0000 7fb8f549b640 1 --2- 172.21.15.7:0/712438518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f009b610 0x7fb8f00ce050 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.359+0000 7fb8f549b640 1 -- 172.21.15.7:0/712438518 >> 172.21.15.7:0/712438518 conn(0x7fb8f00b3f30 msgr2=0x7fb8f00b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:44.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.359+0000 7fb8f549b640 1 -- 172.21.15.7:0/712438518 shutdown_connections 2024-09-15T23:36:44.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.359+0000 7fb8f549b640 1 -- 172.21.15.7:0/712438518 wait complete. 2024-09-15T23:36:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.360+0000 7fb8f549b640 1 Processor -- start 2024-09-15T23:36:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.360+0000 7fb8f549b640 1 -- start start 2024-09-15T23:36:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8f549b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f009b610 0x7fb8f01589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8f549b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8f00ce590 0x7fb8f00c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8f549b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f015e870 0x7fb8f00cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8f549b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f006d170 con 0x7fb8f009b610 2024-09-15T23:36:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8f549b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f00cd550 con 0x7fb8f015e870 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8f549b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f006d2f0 con 0x7fb8f00ce590 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f009b610 0x7fb8f01589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8eeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f009b610 0x7fb8f01589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48122/0 (socket says 172.21.15.7:48122) 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8eeffd640 1 -- 172.21.15.7:0/801998330 learned_addr learned my addr 172.21.15.7:0/801998330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8eeffd640 1 -- 172.21.15.7:0/801998330 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8f00ce590 msgr2=0x7fb8f00c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8ee7fc640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8f00ce590 0x7fb8f00c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.361+0000 7fb8eeffd640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8f00ce590 0x7fb8f00c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8eeffd640 1 -- 172.21.15.7:0/801998330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f015e870 msgr2=0x7fb8f00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8ef7fe640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f015e870 0x7fb8f00cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8eeffd640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f015e870 0x7fb8f00cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8eeffd640 1 -- 172.21.15.7:0/801998330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f0160340 con 0x7fb8f009b610 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8ef7fe640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f015e870 0x7fb8f00cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8eeffd640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f009b610 0x7fb8f01589b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c7c0 tx=0x7fb8e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8e77fe640 1 -- 172.21.15.7:0/801998330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0016020 con 0x7fb8f009b610 2024-09-15T23:36:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8e77fe640 1 -- 172.21.15.7:0/801998330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e0006d50 con 0x7fb8f009b610 2024-09-15T23:36:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f0160620 con 0x7fb8f009b610 2024-09-15T23:36:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8e77fe640 1 -- 172.21.15.7:0/801998330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0003bb0 con 0x7fb8f009b610 2024-09-15T23:36:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.362+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f0160be0 con 0x7fb8f009b610 2024-09-15T23:36:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.363+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f00d14a0 con 0x7fb8f009b610 2024-09-15T23:36:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.363+0000 7fb8e77fe640 1 -- 172.21.15.7:0/801998330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb8e000a850 con 0x7fb8f009b610 2024-09-15T23:36:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.364+0000 7fb8e77fe640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb8d006c2b0 0x7fb8d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.364+0000 7fb8e77fe640 1 -- 172.21.15.7:0/801998330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb8e008b4d0 con 0x7fb8f009b610 2024-09-15T23:36:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.364+0000 7fb8ee7fc640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb8d006c2b0 0x7fb8d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.364+0000 7fb8ee7fc640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb8d006c2b0 0x7fb8d006e770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8034000 tx=0x7fb8d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:44.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.368+0000 7fb8e77fe640 1 -- 172.21.15.7:0/801998330 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb8e0020020 con 0x7fb8f009b610 2024-09-15T23:36:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.589+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb8f0161700 con 0x7fb8f009b610 2024-09-15T23:36:44.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.592+0000 7fb8e77fe640 1 -- 172.21.15.7:0/801998330 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173494 (secure 0 0 0) 0x7fb8e0019310 con 0x7fb8f009b610 2024-09-15T23:36:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:44.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.594+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb8d006c2b0 msgr2=0x7fb8d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.594+0000 7fb8f549b640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb8d006c2b0 0x7fb8d006e770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8034000 tx=0x7fb8d802e000 comp rx=0 tx=0).stop 2024-09-15T23:36:44.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.594+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f009b610 msgr2=0x7fb8f01589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.594+0000 7fb8f549b640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f009b610 0x7fb8f01589b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c7c0 tx=0x7fb8e000cc80 comp rx=0 tx=0).stop 2024-09-15T23:36:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.594+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 shutdown_connections 2024-09-15T23:36:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.594+0000 7fb8f549b640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8f015e870 0x7fb8f00cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.594+0000 7fb8f549b640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb8d006c2b0 0x7fb8d006e770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.594+0000 7fb8f549b640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8f00ce590 0x7fb8f00c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.594+0000 7fb8f549b640 1 --2- 172.21.15.7:0/801998330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f009b610 0x7fb8f01589b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.595+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 >> 172.21.15.7:0/801998330 conn(0x7fb8f00b3f30 msgr2=0x7fb8f00d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.595+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 shutdown_connections 2024-09-15T23:36:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.595+0000 7fb8f549b640 1 -- 172.21.15.7:0/801998330 wait complete. 2024-09-15T23:36:44.607 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3993537392},{"type":"v1","addr":"172.21.15.7:6833","nonce":3993537392}]},"active_addr":"172.21.15.7:6833/3993537392","active_change":"2024-09-15T23:36:03.803979+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":897775139}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":571759858}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":670254998}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":991496469}]}]} 2024-09-15T23:36:44.612 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T23:36:44.612 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T23:36:44.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.712+0000 7fb6e230a640 1 Processor -- start 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.712+0000 7fb6e230a640 1 -- start start 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.713+0000 7fb6e230a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6dc0d51c0 0x7fb6dc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.713+0000 7fb6e230a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 0x7fb6dc0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.713+0000 7fb6e230a640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc0b3d10 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.713+0000 7fb6e230a640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb6dc0d28c0 con 0x7fb6dc0d51c0 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.713+0000 7fb6e230a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc0d3500 con 0x7fb6dc0d30d0 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.713+0000 7fb6db7fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6dc0d51c0 0x7fb6dc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.713+0000 7fb6db7fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6dc0d51c0 0x7fb6dc0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48132/0 (socket says 172.21.15.7:48132) 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.713+0000 7fb6db7fe640 1 -- 172.21.15.7:0/609129098 learned_addr learned my addr 172.21.15.7:0/609129098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.713+0000 7fb6e08a7640 1 --2- 172.21.15.7:0/609129098 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 0x7fb6dc0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.714+0000 7fb6db7fe640 1 -- 172.21.15.7:0/609129098 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 msgr2=0x7fb6dc0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.714+0000 7fb6db7fe640 1 --2- 172.21.15.7:0/609129098 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 0x7fb6dc0b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.714+0000 7fb6db7fe640 1 -- 172.21.15.7:0/609129098 >> v1:172.21.15.18:6789/0 conn(0x7fb6dc15e6d0 legacy=0x7fb6dc0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.714+0000 7fb6db7fe640 1 -- 172.21.15.7:0/609129098 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0cfcd0 con 0x7fb6dc0d51c0 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.714+0000 7fb6e08a7640 1 --2- 172.21.15.7:0/609129098 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 0x7fb6dc0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.714+0000 7fb6db7fe640 1 --2- 172.21.15.7:0/609129098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc0d51c0 0x7fb6dc0d2b90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00098a0 tx=0x7fb6d002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2472c6e544f555e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.714+0000 7fb6daffd640 1 -- 172.21.15.7:0/609129098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0037070 con 0x7fb6dc0d51c0 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6daffd640 1 -- 172.21.15.7:0/609129098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d0032440 con 0x7fb6dc0d51c0 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6daffd640 1 -- 172.21.15.7:0/609129098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0032810 con 0x7fb6dc0d51c0 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6e230a640 1 -- 172.21.15.7:0/609129098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc0d51c0 msgr2=0x7fb6dc0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6e230a640 1 --2- 172.21.15.7:0/609129098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc0d51c0 0x7fb6dc0d2b90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00098a0 tx=0x7fb6d002a290 comp rx=0 tx=0).stop 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6e230a640 1 -- 172.21.15.7:0/609129098 shutdown_connections 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6e230a640 1 --2- 172.21.15.7:0/609129098 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 0x7fb6dc0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6e230a640 1 --2- 172.21.15.7:0/609129098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc0d51c0 0x7fb6dc0d2b90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6e230a640 1 -- 172.21.15.7:0/609129098 >> 172.21.15.7:0/609129098 conn(0x7fb6dc08bfd0 msgr2=0x7fb6dc0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6e230a640 1 -- 172.21.15.7:0/609129098 shutdown_connections 2024-09-15T23:36:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.715+0000 7fb6e230a640 1 -- 172.21.15.7:0/609129098 wait complete. 2024-09-15T23:36:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.716+0000 7fb6e230a640 1 Processor -- start 2024-09-15T23:36:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e230a640 1 -- start start 2024-09-15T23:36:44.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e230a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 0x7fb6dc1615c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e230a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6dc0d51c0 0x7fb6dc1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e230a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc15e6d0 0x7fb6dc0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e230a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc15eab0 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e230a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc0d0230 con 0x7fb6dc0d51c0 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e230a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc0d0510 con 0x7fb6dc0d30d0 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e08a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc15e6d0 0x7fb6dc0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e08a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc15e6d0 0x7fb6dc0d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48138/0 (socket says 172.21.15.7:48138) 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6e08a7640 1 -- 172.21.15.7:0/3285200689 learned_addr learned my addr 172.21.15.7:0/3285200689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6dbfff640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 0x7fb6dc1615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.717+0000 7fb6db7fe640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6dc0d51c0 0x7fb6dc1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6e08a7640 1 -- 172.21.15.7:0/3285200689 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 msgr2=0x7fb6dc1615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6e08a7640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 0x7fb6dc1615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6e08a7640 1 -- 172.21.15.7:0/3285200689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6dc0d51c0 msgr2=0x7fb6dc1509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6e08a7640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6dc0d51c0 0x7fb6dc1509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6e08a7640 1 -- 172.21.15.7:0/3285200689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc161d10 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6e08a7640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc15e6d0 0x7fb6dc0d0a10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009fa0 tx=0x7fb6d400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6d8ff9640 1 -- 172.21.15.7:0/3285200689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d401a070 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6d8ff9640 1 -- 172.21.15.7:0/3285200689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d4003bb0 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6d8ff9640 1 -- 172.21.15.7:0/3285200689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4004030 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6dc0cd900 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.718+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6dc0cdec0 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.719+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6dc0d2b90 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.720+0000 7fb6d8ff9640 1 -- 172.21.15.7:0/3285200689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb6d401d070 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.720+0000 7fb6d8ff9640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb6c806c2b0 0x7fb6c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.720+0000 7fb6d8ff9640 1 -- 172.21.15.7:0/3285200689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb6d408b980 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.720+0000 7fb6dbfff640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb6c806c2b0 0x7fb6c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.721+0000 7fb6dbfff640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb6c806c2b0 0x7fb6c806e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0016e0 tx=0x7fb6cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:44.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.725+0000 7fb6d8ff9640 1 -- 172.21.15.7:0/3285200689 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6d4010070 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.904+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6dc0d1730 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.906+0000 7fb6d8ff9640 1 -- 172.21.15.7:0/3285200689 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7fb6d40597f0 con 0x7fb6dc15e6d0 2024-09-15T23:36:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:44.908 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb6c806c2b0 msgr2=0x7fb6c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb6c806c2b0 0x7fb6c806e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0016e0 tx=0x7fb6cc009040 comp rx=0 tx=0).stop 2024-09-15T23:36:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc15e6d0 msgr2=0x7fb6dc0d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc15e6d0 0x7fb6dc0d0a10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009fa0 tx=0x7fb6d400d560 comp rx=0 tx=0).stop 2024-09-15T23:36:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 shutdown_connections 2024-09-15T23:36:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc15e6d0 0x7fb6dc0d0a10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6dc0d51c0 0x7fb6dc1509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb6c806c2b0 0x7fb6c806e770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 --2- 172.21.15.7:0/3285200689 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb6dc0d30d0 0x7fb6dc1615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 >> 172.21.15.7:0/3285200689 conn(0x7fb6dc08bfd0 msgr2=0x7fb6dc0b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 shutdown_connections 2024-09-15T23:36:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:44.908+0000 7fb6e230a640 1 -- 172.21.15.7:0/3285200689 wait complete. 2024-09-15T23:36:44.922 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T23:36:44.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:45.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.022+0000 7f82f1db7640 1 Processor -- start 2024-09-15T23:36:45.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.022+0000 7f82f1db7640 1 -- start start 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82f1db7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 0x7f82ec0d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82f1db7640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82ec0d0f90 0x7f82ec0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82f1db7640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82ec06dd80 con 0x7f82ec0d52e0 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82f1db7640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f82ec0d1940 con 0x7f82ec0d0f90 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82f1db7640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f82ec0cfe90 con 0x7f82ec15e840 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82ebfff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82ec0d0f90 0x7f82ec0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82eaffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f82ec0d52e0 0x7f82ec06d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44646/0 (socket says 172.21.15.7:44646) 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82eaffd640 1 -- 172.21.15.7:0/2135276324 learned_addr learned my addr 172.21.15.7:0/2135276324 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82eb7fe640 1 --2- 172.21.15.7:0/2135276324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 0x7f82ec0d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82ebfff640 1 -- 172.21.15.7:0/2135276324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 msgr2=0x7f82ec0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82ebfff640 1 --2- 172.21.15.7:0/2135276324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 0x7f82ec0d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82ebfff640 1 -- 172.21.15.7:0/2135276324 >> v1:172.21.15.7:6789/0 conn(0x7f82ec0d52e0 legacy=0x7f82ec06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82ebfff640 1 -- 172.21.15.7:0/2135276324 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec0cf560 con 0x7f82ec0d0f90 2024-09-15T23:36:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82ea7fc640 1 -- 172.21.15.7:0/2135276324 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956686259 0 0) 0x7f82ec06dd80 con 0x7f82ec0d52e0 2024-09-15T23:36:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.023+0000 7f82eb7fe640 1 --2- 172.21.15.7:0/2135276324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 0x7f82ec0d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.024+0000 7f82ebfff640 1 --2- 172.21.15.7:0/2135276324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d0f90 0x7f82ec0d1400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009d90 tx=0x7f82e402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca75a848bb3af7f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.024+0000 7f82ea7fc640 1 -- 172.21.15.7:0/2135276324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4007740 con 0x7f82ec0d0f90 2024-09-15T23:36:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.024+0000 7f82ea7fc640 1 -- 172.21.15.7:0/2135276324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e40078e0 con 0x7f82ec0d0f90 2024-09-15T23:36:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.024+0000 7f82ea7fc640 1 -- 172.21.15.7:0/2135276324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4007cb0 con 0x7f82ec0d0f90 2024-09-15T23:36:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.025+0000 7f82f1db7640 1 -- 172.21.15.7:0/2135276324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d0f90 msgr2=0x7f82ec0d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.025+0000 7f82f1db7640 1 --2- 172.21.15.7:0/2135276324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d0f90 0x7f82ec0d1400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009d90 tx=0x7f82e402a660 comp rx=0 tx=0).stop 2024-09-15T23:36:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.025+0000 7f82f1db7640 1 -- 172.21.15.7:0/2135276324 shutdown_connections 2024-09-15T23:36:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.025+0000 7f82f1db7640 1 --2- 172.21.15.7:0/2135276324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d0f90 0x7f82ec0d1400 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.025+0000 7f82f1db7640 1 --2- 172.21.15.7:0/2135276324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 0x7f82ec0d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.025+0000 7f82f1db7640 1 -- 172.21.15.7:0/2135276324 >> 172.21.15.7:0/2135276324 conn(0x7f82ec08bfd0 msgr2=0x7f82ec0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.025+0000 7f82f1db7640 1 -- 172.21.15.7:0/2135276324 shutdown_connections 2024-09-15T23:36:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.025+0000 7f82f1db7640 1 -- 172.21.15.7:0/2135276324 wait complete. 2024-09-15T23:36:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.026+0000 7f82f1db7640 1 Processor -- start 2024-09-15T23:36:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.026+0000 7f82f1db7640 1 -- start start 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82f1db7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82ec0d0f90 0x7f82ec0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82f1db7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d52e0 0x7f82ec0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82f1db7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 0x7f82ec159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82f1db7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec06e030 con 0x7f82ec0d52e0 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82f1db7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec0cfb20 con 0x7f82ec0d0f90 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82f1db7640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f82ec15ec20 con 0x7f82ec15e840 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82eaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d52e0 0x7f82ec0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82eaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d52e0 0x7f82ec0b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48160/0 (socket says 172.21.15.7:48160) 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82eaffd640 1 -- 172.21.15.7:0/3545045086 learned_addr learned my addr 172.21.15.7:0/3545045086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82eb7fe640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82ec0d0f90 0x7f82ec0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82ebfff640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 0x7f82ec159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82eaffd640 1 -- 172.21.15.7:0/3545045086 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 msgr2=0x7f82ec159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82eaffd640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 0x7f82ec159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82eaffd640 1 -- 172.21.15.7:0/3545045086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82ec0d0f90 msgr2=0x7f82ec0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82eaffd640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82ec0d0f90 0x7f82ec0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.027+0000 7f82eaffd640 1 -- 172.21.15.7:0/3545045086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec0b4560 con 0x7f82ec0d52e0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.028+0000 7f82eaffd640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d52e0 0x7f82ec0b4020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82e000c450 tx=0x7f82e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.028+0000 7f82d3fff640 1 -- 172.21.15.7:0/3545045086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e00074e0 con 0x7f82ec0d52e0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.028+0000 7f82d3fff640 1 -- 172.21.15.7:0/3545045086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e00047e0 con 0x7f82ec0d52e0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.028+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec0b4800 con 0x7f82ec0d52e0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.028+0000 7f82d3fff640 1 -- 172.21.15.7:0/3545045086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0004bb0 con 0x7f82ec0d52e0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.028+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec0b4dc0 con 0x7f82ec0d52e0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.029+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ec0d1400 con 0x7f82ec0d52e0 2024-09-15T23:36:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.029+0000 7f82d3fff640 1 -- 172.21.15.7:0/3545045086 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f82e0011070 con 0x7f82ec0d52e0 2024-09-15T23:36:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.030+0000 7f82d3fff640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f82d806c2b0 0x7f82d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.030+0000 7f82d3fff640 1 -- 172.21.15.7:0/3545045086 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f82e008b240 con 0x7f82ec0d52e0 2024-09-15T23:36:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.030+0000 7f82eb7fe640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f82d806c2b0 0x7f82d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.030+0000 7f82eb7fe640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f82d806c2b0 0x7f82d806e770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82dc006fd0 tx=0x7f82dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.034+0000 7f82d3fff640 1 -- 172.21.15.7:0/3545045086 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f82e0059020 con 0x7f82ec0d52e0 2024-09-15T23:36:45.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.210+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82ec0d1cb0 con 0x7f82ec0d52e0 2024-09-15T23:36:45.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.212+0000 7f82d3fff640 1 -- 172.21.15.7:0/3545045086 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7f82e005d320 con 0x7f82ec0d52e0 2024-09-15T23:36:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:45.214 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.213+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f82d806c2b0 msgr2=0x7f82d806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.213+0000 7f82f1db7640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f82d806c2b0 0x7f82d806e770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82dc006fd0 tx=0x7f82dc008040 comp rx=0 tx=0).stop 2024-09-15T23:36:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.213+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d52e0 msgr2=0x7f82ec0b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.213+0000 7f82f1db7640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d52e0 0x7f82ec0b4020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82e000c450 tx=0x7f82e000c910 comp rx=0 tx=0).stop 2024-09-15T23:36:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.214+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 shutdown_connections 2024-09-15T23:36:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.214+0000 7f82f1db7640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f82d806c2b0 0x7f82d806e770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.214+0000 7f82f1db7640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82ec15e840 0x7f82ec159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.214+0000 7f82f1db7640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec0d52e0 0x7f82ec0b4020 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.214+0000 7f82f1db7640 1 --2- 172.21.15.7:0/3545045086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82ec0d0f90 0x7f82ec0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.214+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 >> 172.21.15.7:0/3545045086 conn(0x7f82ec08bfd0 msgr2=0x7f82ec0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.214+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 shutdown_connections 2024-09-15T23:36:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.214+0000 7f82f1db7640 1 -- 172.21.15.7:0/3545045086 wait complete. 2024-09-15T23:36:45.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T23:36:45.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T23:36:45.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T23:36:45.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T23:36:45.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T23:36:45.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T23:36:45.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T23:36:45.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T23:36:45.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.408+0000 7ff5697d8640 1 Processor -- start 2024-09-15T23:36:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.409+0000 7ff5697d8640 1 -- start start 2024-09-15T23:36:45.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.410+0000 7fd373c9b640 1 Processor -- start 2024-09-15T23:36:45.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.410+0000 7ff5697d8640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5640b0530 0x7ff56406c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.410+0000 7ff5697d8640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5640cdad0 con 0x7ff5640b0a80 2024-09-15T23:36:45.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.410+0000 7ff5697d8640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff56406c990 con 0x7ff564162c30 2024-09-15T23:36:45.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.410+0000 7ff5697d8640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff56409aa40 con 0x7ff5640b0530 2024-09-15T23:36:45.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.411+0000 7ff5637fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5640b0530 0x7ff56406c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.411+0000 7ff5637fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5640b0530 0x7ff56406c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48168/0 (socket says 172.21.15.7:48168) 2024-09-15T23:36:45.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.411+0000 7ff5637fe640 1 -- 172.21.15.7:0/3651245723 learned_addr learned my addr 172.21.15.7:0/3651245723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.411+0000 7f18ce3ea640 1 Processor -- start 2024-09-15T23:36:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.411+0000 7ff5637fe640 1 -- 172.21.15.7:0/3651245723 >> v1:172.21.15.18:6789/0 conn(0x7ff564162c30 legacy=0x7ff56406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.411+0000 7ff5637fe640 1 -- 172.21.15.7:0/3651245723 >> v1:172.21.15.7:6789/0 conn(0x7ff5640b0a80 legacy=0x7ff564070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.411+0000 7ff5637fe640 1 -- 172.21.15.7:0/3651245723 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5640b3900 con 0x7ff5640b0530 2024-09-15T23:36:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.412+0000 7fc81707b640 1 Processor -- start 2024-09-15T23:36:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.411+0000 7fd373c9b640 1 -- start start 2024-09-15T23:36:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.411+0000 7f18ce3ea640 1 -- start start 2024-09-15T23:36:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7f18ce3ea640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f18c0004a20 0x7f18c000bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7ff5637fe640 1 --2- 172.21.15.7:0/3651245723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 0x7ff56406c320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009e80 tx=0x7ff55c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38751574bbd3d008 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7f18ce3ea640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c0055800 con 0x7f18c000a420 2024-09-15T23:36:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7f18ce3ea640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c0006c60 con 0x7f18c0037000 2024-09-15T23:36:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7fd373c9b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c072600 0x7fd36c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7f18ce3ea640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f18c00071d0 con 0x7f18c0004a20 2024-09-15T23:36:45.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7fd373c9b640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd36c06edf0 0x7fd36c0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7fd373c9b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36c06f570 con 0x7fd36c165fa0 2024-09-15T23:36:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7fd373c9b640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fd36c166380 con 0x7fd36c06edf0 2024-09-15T23:36:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7fd373c9b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c06f850 con 0x7fd36c072600 2024-09-15T23:36:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7ff561ffb640 1 -- 172.21.15.7:0/3651245723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c036070 con 0x7ff5640b0530 2024-09-15T23:36:45.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7f18cdbe9640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f18c0037000 0x7f18c00550f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:35386/0 (socket says 172.21.15.7:35386) 2024-09-15T23:36:45.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7f18cdbe9640 1 -- 172.21.15.7:0/3428035124 learned_addr learned my addr 172.21.15.7:0/3428035124 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:45.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.412+0000 7fc81707b640 1 -- start start 2024-09-15T23:36:45.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7fd371236640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c072600 0x7fd36c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7fd371236640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c072600 0x7fd36c06e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:32896/0 (socket says 172.21.15.7:32896) 2024-09-15T23:36:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7fd371236640 1 -- 172.21.15.7:0/1409271029 learned_addr learned my addr 172.21.15.7:0/1409271029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7ff561ffb640 1 -- 172.21.15.7:0/3651245723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c031600 con 0x7ff5640b0530 2024-09-15T23:36:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7f18ccbe7640 1 --2- 172.21.15.7:0/3428035124 >> v2:172.21.15.18:3300/0 conn(0x7f18c0004a20 0x7f18c000bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7f18bffff640 1 -- 172.21.15.7:0/3428035124 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030104314 0 0) 0x7f18c0006c60 con 0x7f18c0037000 2024-09-15T23:36:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.413+0000 7f18bffff640 1 -- 172.21.15.7:0/3428035124 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18b4003520 con 0x7f18c0037000 2024-09-15T23:36:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7f1ab5fd6640 1 Processor -- start 2024-09-15T23:36:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fd371236640 1 -- 172.21.15.7:0/1409271029 >> v2:172.21.15.18:3300/0 conn(0x7fd36c06edf0 msgr2=0x7fd36c0b41d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:36:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fd371236640 1 --2- 172.21.15.7:0/1409271029 >> v2:172.21.15.18:3300/0 conn(0x7fd36c06edf0 0x7fd36c0b41d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fd371236640 1 -- 172.21.15.7:0/1409271029 >> v1:172.21.15.7:6789/0 conn(0x7fd36c165fa0 legacy=0x7fd36c071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fd371236640 1 -- 172.21.15.7:0/1409271029 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c0b39d0 con 0x7fd36c072600 2024-09-15T23:36:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fc81707b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc8100b2570 0x7fc81015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fc81707b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc810154dd0 0x7fc8100b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7f1ab5fd6640 1 -- start start 2024-09-15T23:36:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7ff5697d8640 1 -- 172.21.15.7:0/3651245723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 msgr2=0x7ff56406c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7ff5697d8640 1 --2- 172.21.15.7:0/3651245723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 0x7ff56406c320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009e80 tx=0x7ff55c02a760 comp rx=0 tx=0).stop 2024-09-15T23:36:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7ff5697d8640 1 -- 172.21.15.7:0/3651245723 shutdown_connections 2024-09-15T23:36:45.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7ff5697d8640 1 --2- 172.21.15.7:0/3651245723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 0x7ff56406c320 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7ff5697d8640 1 -- 172.21.15.7:0/3651245723 >> 172.21.15.7:0/3651245723 conn(0x7ff5640734b0 msgr2=0x7ff5640b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7fadcbef1640 1 Processor -- start 2024-09-15T23:36:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fc81707b640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8100b36a0 con 0x7fc81015e3a0 2024-09-15T23:36:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fc81707b640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc8100732e0 con 0x7fc810154dd0 2024-09-15T23:36:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fc81707b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fc810073e60 con 0x7fc8100b2570 2024-09-15T23:36:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fc815618640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fc81015e3a0 0x7fc81015e7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:35398/0 (socket says 172.21.15.7:35398) 2024-09-15T23:36:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7ff5697d8640 1 -- 172.21.15.7:0/3651245723 shutdown_connections 2024-09-15T23:36:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f151e4f1640 1 Processor -- start 2024-09-15T23:36:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f151e4f1640 1 -- start start 2024-09-15T23:36:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fc80ffff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc810154dd0 0x7fc8100b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7fc815618640 1 -- 172.21.15.7:0/1028761560 learned_addr learned my addr 172.21.15.7:0/1028761560 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7fadcbef1640 1 -- start start 2024-09-15T23:36:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7fc80ffff640 1 -- 172.21.15.7:0/1028761560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc8100b2570 msgr2=0x7fc81015b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:36:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7fc80ffff640 1 --2- 172.21.15.7:0/1028761560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc8100b2570 0x7fc81015b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7fc80ffff640 1 -- 172.21.15.7:0/1028761560 >> v1:172.21.15.18:6789/0 conn(0x7fc81015e3a0 legacy=0x7fc81015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7fc80ffff640 1 -- 172.21.15.7:0/1028761560 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc810073530 con 0x7fc810154dd0 2024-09-15T23:36:45.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f1ab5fd6640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ab00d3140 0x7f1ab00d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7f18ccbe7640 1 -- 172.21.15.7:0/3428035124 >> v1:172.21.15.18:6789/0 conn(0x7f18c0037000 legacy=0x7f18c00550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7f18ccbe7640 1 -- 172.21.15.7:0/3428035124 >> v1:172.21.15.7:6789/0 conn(0x7f18c000a420 legacy=0x7f18c0039690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.414+0000 7f18ccbe7640 1 -- 172.21.15.7:0/3428035124 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c0007420 con 0x7f18c0004a20 2024-09-15T23:36:45.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f18ccbe7640 1 --2- 172.21.15.7:0/3428035124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18c0004a20 0x7f18c000bbb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18c40097a0 tx=0x7f18c402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70408c327a68197c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f1ab5fd6640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1ab00d1290 0x7f1ab006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f1ab5fd6640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab00d03d0 con 0x7f1ab015e700 2024-09-15T23:36:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f1ab5fd6640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab00d2cb0 con 0x7f1ab00d3140 2024-09-15T23:36:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f1ab5fd6640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab00d16c0 con 0x7f1ab00d1290 2024-09-15T23:36:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7fd371236640 1 --2- 172.21.15.7:0/1409271029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c072600 0x7fd36c06e8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009870 tx=0x7fd35c02a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dc3b7af8196c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f1aaffff640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1ab00d1290 0x7f1ab006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f31da01e640 1 Processor -- start 2024-09-15T23:36:45.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.415+0000 7f31da01e640 1 -- start start 2024-09-15T23:36:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f1aaffff640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1ab00d1290 0x7f1ab006cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:47136/0 (socket says 172.21.15.7:47136) 2024-09-15T23:36:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f1aaffff640 1 -- 172.21.15.7:0/1276878319 learned_addr learned my addr 172.21.15.7:0/1276878319 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7fd370a35640 1 -- 172.21.15.7:0/1409271029 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c037070 con 0x7fd36c072600 2024-09-15T23:36:45.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7fd370a35640 1 -- 172.21.15.7:0/1409271029 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c032440 con 0x7fd36c072600 2024-09-15T23:36:45.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7fd370a35640 1 -- 172.21.15.7:0/1409271029 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c032810 con 0x7fd36c072600 2024-09-15T23:36:45.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f1aaffff640 1 -- 172.21.15.7:0/1276878319 >> v2:172.21.15.7:3300/0 conn(0x7f1ab00d3140 msgr2=0x7f1ab00d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f1aaffff640 1 --2- 172.21.15.7:0/1276878319 >> v2:172.21.15.7:3300/0 conn(0x7f1ab00d3140 0x7f1ab00d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f1aaffff640 1 -- 172.21.15.7:0/1276878319 >> v1:172.21.15.7:6789/0 conn(0x7f1ab015e700 legacy=0x7f1ab00cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f1aaffff640 1 -- 172.21.15.7:0/1276878319 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab00d2380 con 0x7f1ab00d1290 2024-09-15T23:36:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f18bffff640 1 -- 172.21.15.7:0/3428035124 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4037070 con 0x7f18c0004a20 2024-09-15T23:36:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f18bffff640 1 -- 172.21.15.7:0/3428035124 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c4032440 con 0x7f18c0004a20 2024-09-15T23:36:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f18bffff640 1 -- 172.21.15.7:0/3428035124 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4032810 con 0x7f18c0004a20 2024-09-15T23:36:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7f151e4f1640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f151806cd40 0x7f15180d4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7fadcbef1640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fadc4072420 0x7fadc40cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.416+0000 7fadcbef1640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fadc4162ab0 0x7fadc406f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fadcbef1640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40b3970 con 0x7fadc4072800 2024-09-15T23:36:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fadcbef1640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fadc40b3c50 con 0x7fadc4072420 2024-09-15T23:36:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fadcbef1640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fadc40b3f30 con 0x7fadc4162ab0 2024-09-15T23:36:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f1517fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f151806cd40 0x7f15180d4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f1517fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f151806cd40 0x7f15180d4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48214/0 (socket says 172.21.15.7:48214) 2024-09-15T23:36:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fd373c9b640 1 -- 172.21.15.7:0/1409271029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c072600 msgr2=0x7fd36c06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fd373c9b640 1 --2- 172.21.15.7:0/1409271029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c072600 0x7fd36c06e8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009870 tx=0x7fd35c02a230 comp rx=0 tx=0).stop 2024-09-15T23:36:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7ff5697d8640 1 -- 172.21.15.7:0/3651245723 wait complete. 2024-09-15T23:36:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fc80ffff640 1 --2- 172.21.15.7:0/1028761560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc810154dd0 0x7fc8100b3160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc800013db0 tx=0x7fc8000327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24265ca3947f1a24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fadca48e640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fadc4072800 0x7fadc4166440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:35422/0 (socket says 172.21.15.7:35422) 2024-09-15T23:36:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fadca48e640 1 -- 172.21.15.7:0/3697346468 learned_addr learned my addr 172.21.15.7:0/3697346468 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fadc9c8d640 1 --2- 172.21.15.7:0/3697346468 >> v2:172.21.15.7:3300/0 conn(0x7fadc4072420 0x7fadc40cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fd373c9b640 1 -- 172.21.15.7:0/1409271029 shutdown_connections 2024-09-15T23:36:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fd373c9b640 1 --2- 172.21.15.7:0/1409271029 >> v2:172.21.15.18:3300/0 conn(0x7fd36c06edf0 0x7fd36c0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fd373c9b640 1 --2- 172.21.15.7:0/1409271029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c072600 0x7fd36c06e8b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f15177fe640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f151806d120 0x7f1518162710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:35406/0 (socket says 172.21.15.7:35406) 2024-09-15T23:36:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fd373c9b640 1 -- 172.21.15.7:0/1409271029 >> 172.21.15.7:0/1409271029 conn(0x7fd36c0b1e60 msgr2=0x7fd36c0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f1aaffff640 1 --2- 172.21.15.7:0/1276878319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab00d1290 0x7f1ab006cdd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00c7ee0 tx=0x7f1aa402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9366a8c0240bb92 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f31da01e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31cc0ea920 0x7f31cc038890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fd373c9b640 1 -- 172.21.15.7:0/1409271029 shutdown_connections 2024-09-15T23:36:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f1517fff640 1 -- 172.21.15.7:0/2361801718 learned_addr learned my addr 172.21.15.7:0/2361801718 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f151e4f1640 1 --2- 172.21.15.7:0/2361801718 >> v2:172.21.15.18:3300/0 conn(0x7f151806e560 0x7f1518162e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f18ce3ea640 1 -- 172.21.15.7:0/3428035124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18c0004a20 msgr2=0x7f18c000bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f18ce3ea640 1 --2- 172.21.15.7:0/3428035124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18c0004a20 0x7f18c000bbb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18c40097a0 tx=0x7f18c402a270 comp rx=0 tx=0).stop 2024-09-15T23:36:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f151e4f1640 1 -- 172.21.15.7:0/2361801718 --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151806e940 con 0x7f151806d120 2024-09-15T23:36:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f1aae7fc640 1 -- 172.21.15.7:0/1276878319 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4037070 con 0x7f1ab00d1290 2024-09-15T23:36:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f151e4f1640 1 -- 172.21.15.7:0/2361801718 --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f151806e0d0 con 0x7f151806cd40 2024-09-15T23:36:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f151e4f1640 1 -- 172.21.15.7:0/2361801718 --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f151806d7a0 con 0x7f151806e560 2024-09-15T23:36:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f151ca8e640 1 --2- 172.21.15.7:0/2361801718 >> v2:172.21.15.18:3300/0 conn(0x7f151806e560 0x7f1518162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f1aae7fc640 1 -- 172.21.15.7:0/1276878319 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa4032440 con 0x7f1ab00d1290 2024-09-15T23:36:45.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f1aae7fc640 1 -- 172.21.15.7:0/1276878319 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4032810 con 0x7f1ab00d1290 2024-09-15T23:36:45.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7fadc948c640 1 --2- 172.21.15.7:0/3697346468 >> v2:172.21.15.18:3300/0 conn(0x7fadc4162ab0 0x7fadc406f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7fadc9c8d640 1 -- 172.21.15.7:0/3697346468 >> v2:172.21.15.18:3300/0 conn(0x7fadc4162ab0 msgr2=0x7fadc406f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7fadc9c8d640 1 --2- 172.21.15.7:0/3697346468 >> v2:172.21.15.18:3300/0 conn(0x7fadc4162ab0 0x7fadc406f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7fadc9c8d640 1 -- 172.21.15.7:0/3697346468 >> v1:172.21.15.18:6789/0 conn(0x7fadc4072800 legacy=0x7fadc4166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7fadc8c8b640 1 -- 172.21.15.7:0/3697346468 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3112863890 0 0) 0x7fadc40b3970 con 0x7fadc4072800 2024-09-15T23:36:45.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f1516ffd640 1 -- 172.21.15.7:0/2361801718 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923267712 0 0) 0x7f151806e940 con 0x7f151806d120 2024-09-15T23:36:45.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f1516ffd640 1 -- 172.21.15.7:0/2361801718 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1500003520 con 0x7f151806d120 2024-09-15T23:36:45.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7fc80f7fe640 1 -- 172.21.15.7:0/1028761560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80003e070 con 0x7fc810154dd0 2024-09-15T23:36:45.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7fc80f7fe640 1 -- 172.21.15.7:0/1028761560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc800037710 con 0x7fc810154dd0 2024-09-15T23:36:45.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7fc80f7fe640 1 -- 172.21.15.7:0/1028761560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800037ae0 con 0x7fc810154dd0 2024-09-15T23:36:45.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7fadc9c8d640 1 -- 172.21.15.7:0/3697346468 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc40b4210 con 0x7fadc4072420 2024-09-15T23:36:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f1517fff640 1 -- 172.21.15.7:0/2361801718 >> v2:172.21.15.18:3300/0 conn(0x7f151806e560 msgr2=0x7f1518162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f1517fff640 1 --2- 172.21.15.7:0/2361801718 >> v2:172.21.15.18:3300/0 conn(0x7f151806e560 0x7f1518162e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f1517fff640 1 -- 172.21.15.7:0/2361801718 >> v1:172.21.15.18:6789/0 conn(0x7f151806d120 legacy=0x7f1518162710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f1517fff640 1 -- 172.21.15.7:0/2361801718 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151806da80 con 0x7f151806cd40 2024-09-15T23:36:45.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f18ce3ea640 1 -- 172.21.15.7:0/3428035124 shutdown_connections 2024-09-15T23:36:45.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f18ce3ea640 1 --2- 172.21.15.7:0/3428035124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18c0004a20 0x7f18c000bbb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f18ce3ea640 1 -- 172.21.15.7:0/3428035124 >> 172.21.15.7:0/3428035124 conn(0x7f18c001dd70 msgr2=0x7f18c00549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7fd373c9b640 1 -- 172.21.15.7:0/1409271029 wait complete. 2024-09-15T23:36:45.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f1ab5fd6640 1 -- 172.21.15.7:0/1276878319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab00d1290 msgr2=0x7f1ab006cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f1ab5fd6640 1 --2- 172.21.15.7:0/1276878319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab00d1290 0x7f1ab006cdd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00c7ee0 tx=0x7f1aa402a2c0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7ff5697d8640 1 Processor -- start 2024-09-15T23:36:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f1ab5fd6640 1 -- 172.21.15.7:0/1276878319 shutdown_connections 2024-09-15T23:36:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f1ab5fd6640 1 --2- 172.21.15.7:0/1276878319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab00d1290 0x7f1ab006cdd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f1ab5fd6640 1 --2- 172.21.15.7:0/1276878319 >> v2:172.21.15.7:3300/0 conn(0x7f1ab00d3140 0x7f1ab00d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f1ab5fd6640 1 -- 172.21.15.7:0/1276878319 >> 172.21.15.7:0/1276878319 conn(0x7f1ab008bfd0 msgr2=0x7f1ab00d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fc81707b640 1 -- 172.21.15.7:0/1028761560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc810154dd0 msgr2=0x7fc8100b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fc81707b640 1 --2- 172.21.15.7:0/1028761560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc810154dd0 0x7fc8100b3160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc800013db0 tx=0x7fc8000327b0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f31da01e640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31cc006850 con 0x7f31cc051f90 2024-09-15T23:36:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f31da01e640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31cc034340 con 0x7f31cc033b80 2024-09-15T23:36:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f31da01e640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f31cc007700 con 0x7f31cc0ea920 2024-09-15T23:36:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f1ab5fd6640 1 -- 172.21.15.7:0/1276878319 shutdown_connections 2024-09-15T23:36:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f31d2ffd640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f31cc033b80 0x7f31cc051880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:35432/0 (socket says 172.21.15.7:35432) 2024-09-15T23:36:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.417+0000 7f31d2ffd640 1 -- 172.21.15.7:0/2662303201 learned_addr learned my addr 172.21.15.7:0/2662303201 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3830835123 0 0) 0x7f31cc034340 con 0x7f31cc033b80 2024-09-15T23:36:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31c4003520 con 0x7f31cc033b80 2024-09-15T23:36:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 <== mon.1 v1:172.21.15.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2641303583 0 0) 0x7f31c4003520 con 0x7f31cc033b80 2024-09-15T23:36:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 >> v2:172.21.15.7:3300/0 conn(0x7f31cc0ea920 msgr2=0x7f31cc038890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f31d27fc640 1 --2- 172.21.15.7:0/2662303201 >> v2:172.21.15.7:3300/0 conn(0x7f31cc0ea920 0x7f31cc038890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 >> v1:172.21.15.7:6789/0 conn(0x7f31cc051f90 legacy=0x7f31cc00a420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:36:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 --> v1:172.21.15.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc006af0 con 0x7f31cc033b80 2024-09-15T23:36:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 <== mon.1 v1:172.21.15.18:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f31c8001670 con 0x7f31cc033b80 2024-09-15T23:36:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.418+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 >> v1:172.21.15.18:6789/0 conn(0x7f31cc033b80 legacy=0x7f31cc051880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f31d27fc640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4003a00 0x7f31c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f31d27fc640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 0x7f31c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f31d27fc640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31c40052e0 0x7f31c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc007700 con 0x7f31c4004800 2024-09-15T23:36:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f31c400c230 con 0x7f31c4003a00 2024-09-15T23:36:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f31d2ffd640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 0x7f31c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f31c400c430 con 0x7f31c40052e0 2024-09-15T23:36:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f18ce3ea640 1 -- 172.21.15.7:0/3428035124 shutdown_connections 2024-09-15T23:36:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fc81707b640 1 -- 172.21.15.7:0/1028761560 shutdown_connections 2024-09-15T23:36:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fc81707b640 1 --2- 172.21.15.7:0/1028761560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc810154dd0 0x7fc8100b3160 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fc81707b640 1 --2- 172.21.15.7:0/1028761560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc8100b2570 0x7fc81015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fc81707b640 1 -- 172.21.15.7:0/1028761560 >> 172.21.15.7:0/1028761560 conn(0x7fc810072430 msgr2=0x7fc8100b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fadc9c8d640 1 --2- 172.21.15.7:0/3697346468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc4072420 0x7fadc40cfef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fadc40b32a0 tx=0x7fadc002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd315d7eb3207717 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fc81707b640 1 -- 172.21.15.7:0/1028761560 shutdown_connections 2024-09-15T23:36:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f1517fff640 1 --2- 172.21.15.7:0/2361801718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151806cd40 0x7f15180d4b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f151806e250 tx=0x7f151002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e190f8d4634aa66 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7ff5697d8640 1 -- start start 2024-09-15T23:36:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fadc8c8b640 1 -- 172.21.15.7:0/3697346468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0037070 con 0x7fadc4072420 2024-09-15T23:36:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7fd373c9b640 1 Processor -- start 2024-09-15T23:36:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f31d37fe640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4003a00 0x7f31c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7f31d3fff640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31c40052e0 0x7f31c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7f1516ffd640 1 -- 172.21.15.7:0/2361801718 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510036070 con 0x7f151806cd40 2024-09-15T23:36:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7f1516ffd640 1 -- 172.21.15.7:0/2361801718 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1510031600 con 0x7f151806cd40 2024-09-15T23:36:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7f1516ffd640 1 -- 172.21.15.7:0/2361801718 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15100319d0 con 0x7f151806cd40 2024-09-15T23:36:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7f31d2ffd640 1 -- 172.21.15.7:0/2662303201 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31c40052e0 msgr2=0x7f31c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7fd373c9b640 1 -- start start 2024-09-15T23:36:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7f31d2ffd640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31c40052e0 0x7f31c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7f31d2ffd640 1 -- 172.21.15.7:0/2662303201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4003a00 msgr2=0x7f31c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7f31d2ffd640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4003a00 0x7f31c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7f31d2ffd640 1 -- 172.21.15.7:0/2662303201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c4003520 con 0x7f31c4004800 2024-09-15T23:36:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7ff5697d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 0x7ff5640ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7ff5697d8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5640b0a80 0x7ff56414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7fadc8c8b640 1 -- 172.21.15.7:0/3697346468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc0032440 con 0x7fadc4072420 2024-09-15T23:36:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7fadcbef1640 1 -- 172.21.15.7:0/3697346468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc4072420 msgr2=0x7fadc40cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 0x7ff5640ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7fc81707b640 1 -- 172.21.15.7:0/1028761560 wait complete. 2024-09-15T23:36:45.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fadc8c8b640 1 -- 172.21.15.7:0/3697346468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0037070 con 0x7fadc4072420 2024-09-15T23:36:45.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fadcbef1640 1 --2- 172.21.15.7:0/3697346468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc4072420 0x7fadc40cfef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fadc40b32a0 tx=0x7fadc002a110 comp rx=0 tx=0).stop 2024-09-15T23:36:45.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.420+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 0x7ff5640ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48262/0 (socket says 172.21.15.7:48262) 2024-09-15T23:36:45.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fd373c9b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c06edf0 0x7fd36c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fadcbef1640 1 -- 172.21.15.7:0/3697346468 shutdown_connections 2024-09-15T23:36:45.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fadcbef1640 1 --2- 172.21.15.7:0/3697346468 >> v2:172.21.15.18:3300/0 conn(0x7fadc4162ab0 0x7fadc406f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fadcbef1640 1 --2- 172.21.15.7:0/3697346468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc4072420 0x7fadc40cfef0 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fadc40b32a0 tx=0x7fadc002a110 comp rx=0 tx=0).stop 2024-09-15T23:36:45.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fadcbef1640 1 -- 172.21.15.7:0/3697346468 >> 172.21.15.7:0/3697346468 conn(0x7fadc40b25d0 msgr2=0x7fadc40b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff5697d8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff564162c30 0x7ff5640b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff5697d8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff564163190 con 0x7ff5640b0530 2024-09-15T23:36:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff5697d8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff564163010 con 0x7ff5640b0a80 2024-09-15T23:36:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff5697d8640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff5640b62c0 con 0x7ff564162c30 2024-09-15T23:36:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff562ffd640 1 -- 172.21.15.7:0/1923736179 learned_addr learned my addr 172.21.15.7:0/1923736179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f18ce3ea640 1 -- 172.21.15.7:0/3428035124 wait complete. 2024-09-15T23:36:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.419+0000 7f1ab5fd6640 1 -- 172.21.15.7:0/1276878319 wait complete. 2024-09-15T23:36:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7f18ce3ea640 1 Processor -- start 2024-09-15T23:36:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fd371a37640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c06edf0 0x7fd36c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fd371a37640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c06edf0 0x7fd36c15f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:32908/0 (socket says 172.21.15.7:32908) 2024-09-15T23:36:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff5627fc640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5640b0a80 0x7ff56414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fd371a37640 1 -- 172.21.15.7:0/798043146 learned_addr learned my addr 172.21.15.7:0/798043146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fc81707b640 1 Processor -- start 2024-09-15T23:36:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fadcbef1640 1 -- 172.21.15.7:0/3697346468 shutdown_connections 2024-09-15T23:36:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fd373c9b640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd36c072600 0x7fd36c09c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fd373c9b640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd36c165fa0 0x7fd36c09c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fd373c9b640 1 -- 172.21.15.7:0/798043146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c06f260 con 0x7fd36c072600 2024-09-15T23:36:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fd373c9b640 1 -- 172.21.15.7:0/798043146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c0b36d0 con 0x7fd36c165fa0 2024-09-15T23:36:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fd373c9b640 1 -- 172.21.15.7:0/798043146 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c0ad140 con 0x7fd36c06edf0 2024-09-15T23:36:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7f18ce3ea640 1 -- start start 2024-09-15T23:36:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff562ffd640 1 -- 172.21.15.7:0/1923736179 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff564162c30 msgr2=0x7ff5640b15f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:36:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff562ffd640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff564162c30 0x7ff5640b15f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff562ffd640 1 -- 172.21.15.7:0/1923736179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5640b0a80 msgr2=0x7ff56414fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff562ffd640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5640b0a80 0x7ff56414fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7ff562ffd640 1 -- 172.21.15.7:0/1923736179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5640b2f50 con 0x7ff5640b0530 2024-09-15T23:36:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7fc81707b640 1 -- start start 2024-09-15T23:36:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f151e4f1640 1 -- 172.21.15.7:0/2361801718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151806cd40 msgr2=0x7f15180d4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7fd372238640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd36c165fa0 0x7fd36c09c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f151e4f1640 1 --2- 172.21.15.7:0/2361801718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151806cd40 0x7f15180d4b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f151806e250 tx=0x7f151002a6a0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f151e4f1640 1 -- 172.21.15.7:0/2361801718 shutdown_connections 2024-09-15T23:36:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f151e4f1640 1 --2- 172.21.15.7:0/2361801718 >> v2:172.21.15.18:3300/0 conn(0x7f151806e560 0x7f1518162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f151e4f1640 1 --2- 172.21.15.7:0/2361801718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151806cd40 0x7f15180d4b90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f151e4f1640 1 -- 172.21.15.7:0/2361801718 >> 172.21.15.7:0/2361801718 conn(0x7f151808bfd0 msgr2=0x7f15180d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7fd371a37640 1 -- 172.21.15.7:0/798043146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd36c165fa0 msgr2=0x7fd36c09c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7fd371a37640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd36c165fa0 0x7fd36c09c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7fd371a37640 1 -- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd36c072600 msgr2=0x7fd36c09c0a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:36:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7fd371a37640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd36c072600 0x7fd36c09c0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f151e4f1640 1 -- 172.21.15.7:0/2361801718 shutdown_connections 2024-09-15T23:36:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f151e4f1640 1 -- 172.21.15.7:0/2361801718 wait complete. 2024-09-15T23:36:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7ff562ffd640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 0x7ff5640ceff0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c400 tx=0x7ff55400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7ff54b7fe640 1 -- 172.21.15.7:0/1923736179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554017070 con 0x7ff5640b0530 2024-09-15T23:36:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7fd371a37640 1 -- 172.21.15.7:0/798043146 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c1eb7a0 con 0x7fd36c06edf0 2024-09-15T23:36:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7fd371a37640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c06edf0 0x7fd36c15f860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b3e0 tx=0x7fd36800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f18ce3ea640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f18c0004a20 0x7f18c0188540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7fd3627fc640 1 -- 172.21.15.7:0/798043146 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368015020 con 0x7fd36c06edf0 2024-09-15T23:36:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5640b3230 con 0x7ff5640b0530 2024-09-15T23:36:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5640cf530 con 0x7ff5640b0530 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f18ce3ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18c000a420 0x7f18c00528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f18ce3ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c0037000 0x7f18c004e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f18cd3e8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f18c0004a20 0x7f18c0188540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f18cd3e8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f18c0004a20 0x7f18c0188540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:32918/0 (socket says 172.21.15.7:32918) 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f18ccbe7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18c000a420 0x7f18c00528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f18ccbe7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18c000a420 0x7f18c00528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:47190/0 (socket says 172.21.15.7:47190) 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f18cdbe9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c0037000 0x7f18c004e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18cdbe9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c0037000 0x7f18c004e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48288/0 (socket says 172.21.15.7:48288) 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18cd3e8640 1 -- 172.21.15.7:0/164654938 learned_addr learned my addr 172.21.15.7:0/164654938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0006a20 con 0x7f18c0037000 2024-09-15T23:36:45.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18c00068a0 con 0x7f18c000a420 2024-09-15T23:36:45.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f18c00debd0 con 0x7f18c0004a20 2024-09-15T23:36:45.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7fc81707b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8100b2570 0x7fc81009a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7fc81707b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc810154dd0 0x7fc8101ed710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fc814e17640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8100b2570 0x7fc81009a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fc814e17640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8100b2570 0x7fc81009a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:47202/0 (socket says 172.21.15.7:47202) 2024-09-15T23:36:45.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fc814e17640 1 -- 172.21.15.7:0/917384607 learned_addr learned my addr 172.21.15.7:0/917384607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fc81707b640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc81015e3a0 0x7fc8101f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7fadcbef1640 1 -- 172.21.15.7:0/3697346468 wait complete. 2024-09-15T23:36:45.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7fadcbef1640 1 Processor -- start 2024-09-15T23:36:45.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7f31d3fff640 1 -- 172.21.15.7:0/2662303201 reap_dead start 2024-09-15T23:36:45.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7f31d2ffd640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 0x7f31c4004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31c800c160 tx=0x7f31c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a868b954c51e89dc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8007c60 con 0x7f31c4004800 2024-09-15T23:36:45.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c8005ad0 con 0x7f31c4004800 2024-09-15T23:36:45.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f31d27fc640 1 -- 172.21.15.7:0/2662303201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8007070 con 0x7f31c4004800 2024-09-15T23:36:45.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f31da01e640 1 -- 172.21.15.7:0/2662303201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 msgr2=0x7f31c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f31da01e640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 0x7f31c4004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31c800c160 tx=0x7f31c4005830 comp rx=0 tx=0).stop 2024-09-15T23:36:45.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f151e4f1640 1 Processor -- start 2024-09-15T23:36:45.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc810155230 con 0x7fc810154dd0 2024-09-15T23:36:45.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc810073af0 con 0x7fc8100b2570 2024-09-15T23:36:45.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f31da01e640 1 -- 172.21.15.7:0/2662303201 shutdown_connections 2024-09-15T23:36:45.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f31da01e640 1 --2- 172.21.15.7:0/2662303201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 0x7f31c4004c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7fd3627fc640 1 -- 172.21.15.7:0/798043146 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd368011850 con 0x7fd36c06edf0 2024-09-15T23:36:45.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7fd3627fc640 1 -- 172.21.15.7:0/798043146 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368011c40 con 0x7fd36c06edf0 2024-09-15T23:36:45.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f31da01e640 1 -- 172.21.15.7:0/2662303201 >> 172.21.15.7:0/2662303201 conn(0x7f31cc01e1c0 msgr2=0x7f31cc037dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fd373c9b640 1 -- 172.21.15.7:0/798043146 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c1eba80 con 0x7fd36c06edf0 2024-09-15T23:36:45.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f151e4f1640 1 -- start start 2024-09-15T23:36:45.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5640b0ef0 con 0x7ff5640b0530 2024-09-15T23:36:45.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.421+0000 7f1ab5fd6640 1 Processor -- start 2024-09-15T23:36:45.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f31da01e640 1 -- 172.21.15.7:0/2662303201 shutdown_connections 2024-09-15T23:36:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.422+0000 7f1ab5fd6640 1 -- start start 2024-09-15T23:36:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fd373c9b640 1 -- 172.21.15.7:0/798043146 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c1ec040 con 0x7fd36c06edf0 2024-09-15T23:36:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18cd3e8640 1 -- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18c000a420 msgr2=0x7f18c00528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18cd3e8640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18c000a420 0x7f18c00528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18cd3e8640 1 -- 172.21.15.7:0/164654938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c0037000 msgr2=0x7f18c004e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18cd3e8640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c0037000 0x7f18c004e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f18cd3e8640 1 -- 172.21.15.7:0/164654938 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c004ba20 con 0x7f18c0004a20 2024-09-15T23:36:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7f31da01e640 1 -- 172.21.15.7:0/2662303201 wait complete. 2024-09-15T23:36:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f1ab5fd6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1ab00d1290 0x7f1ab0157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fc8100b0ce0 con 0x7fc81015e3a0 2024-09-15T23:36:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fc80ffff640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc810154dd0 0x7fc8101ed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f1ab5fd6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab00d3140 0x7f1ab0203ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fc815618640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc81015e3a0 0x7fc8101f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7f18cd3e8640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f18c0004a20 0x7f18c0188540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f18c802dd50 tx=0x7f18c8069e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7fadcbef1640 1 -- start start 2024-09-15T23:36:45.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f1ab5fd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab015e700 0x7f1ab0158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadcbef1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc4072800 0x7fadc4159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadcbef1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc4162ab0 0x7fadc4201c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f1ab5fd6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab015eae0 con 0x7f1ab00d3140 2024-09-15T23:36:45.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadcbef1640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fadc415cc80 0x7fadc415dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7f18bdffb640 1 -- 172.21.15.7:0/164654938 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c80741c0 con 0x7f18c0004a20 2024-09-15T23:36:45.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadcbef1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4072d90 con 0x7fadc4072800 2024-09-15T23:36:45.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadcbef1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4072c10 con 0x7fadc4162ab0 2024-09-15T23:36:45.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadcbef1640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fadc40b4730 con 0x7fadc415cc80 2024-09-15T23:36:45.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7f18bdffb640 1 -- 172.21.15.7:0/164654938 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c8069fa0 con 0x7f18c0004a20 2024-09-15T23:36:45.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f1ab5fd6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab00d2940 con 0x7f1ab015e700 2024-09-15T23:36:45.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f151e4f1640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f151806cd40 0x7f15180cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f151e4f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151806d120 0x7f15180c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f1517fff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f151806cd40 0x7f15180cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f1517fff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f151806cd40 0x7f15180cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:32938/0 (socket says 172.21.15.7:32938) 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f1517fff640 1 -- 172.21.15.7:0/3246347815 learned_addr learned my addr 172.21.15.7:0/3246347815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f151e4f1640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151806e560 0x7f1518200fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f1ab5fd6640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab00d3580 con 0x7f1ab00d1290 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.423+0000 7f1aaeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab00d3140 0x7f1ab0203ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f1aaeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab00d3140 0x7f1ab0203ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48278/0 (socket says 172.21.15.7:48278) 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f1aaeffd640 1 -- 172.21.15.7:0/210916152 learned_addr learned my addr 172.21.15.7:0/210916152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f1aaeffd640 1 -- 172.21.15.7:0/210916152 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1ab00d1290 msgr2=0x7f1ab0157f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f1aaeffd640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1ab00d1290 0x7f1ab0157f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f1aaeffd640 1 -- 172.21.15.7:0/210916152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab015e700 msgr2=0x7f1ab0158670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:36:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f1aaeffd640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab015e700 0x7f1ab0158670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.424+0000 7f1aaeffd640 1 -- 172.21.15.7:0/210916152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab00b62b0 con 0x7f1ab00d3140 2024-09-15T23:36:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7f1aaeffd640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab00d3140 0x7f1ab0203ec0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800bd90 tx=0x7f1aa8009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7f1a97fff640 1 -- 172.21.15.7:0/210916152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8019070 con 0x7f1ab00d3140 2024-09-15T23:36:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7f1ab5fd6640 1 -- 172.21.15.7:0/210916152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab00b6590 con 0x7f1ab00d3140 2024-09-15T23:36:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7f18bdffb640 1 -- 172.21.15.7:0/164654938 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8063270 con 0x7f18c0004a20 2024-09-15T23:36:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fc80ffff640 1 -- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc81015e3a0 msgr2=0x7fc8101f0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fc80ffff640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc81015e3a0 0x7fc8101f0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fc80ffff640 1 -- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8100b2570 msgr2=0x7fc81009a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fc80ffff640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8100b2570 0x7fc81009a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fc80ffff640 1 -- 172.21.15.7:0/917384607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8100af720 con 0x7fc810154dd0 2024-09-15T23:36:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7fc80ffff640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc810154dd0 0x7fc8101ed710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc8100d1220 tx=0x7fc8000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7fc815618640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc81015e3a0 0x7fc8101f0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadc948c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc4162ab0 0x7fadc4201c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadca48e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fadc415cc80 0x7fadc415dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadca48e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fadc415cc80 0x7fadc415dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:32936/0 (socket says 172.21.15.7:32936) 2024-09-15T23:36:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadca48e640 1 -- 172.21.15.7:0/2426920566 learned_addr learned my addr 172.21.15.7:0/2426920566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7fadc9c8d640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc4072800 0x7fadc4159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f15177fe640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151806d120 0x7f15180c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f151ca8e640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151806e560 0x7f1518200fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7fadc9c8d640 1 -- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fadc415cc80 msgr2=0x7fadc415dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7fadc9c8d640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fadc415cc80 0x7fadc415dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7fadc9c8d640 1 -- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc4162ab0 msgr2=0x7fadc4201c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7fadc9c8d640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc4162ab0 0x7fadc4201c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1518163360 con 0x7f151806d120 2024-09-15T23:36:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f15180d30c0 con 0x7f151806e560 2024-09-15T23:36:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7fadc9c8d640 1 -- 172.21.15.7:0/2426920566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc4202160 con 0x7fadc4072800 2024-09-15T23:36:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f15181634e0 con 0x7f151806cd40 2024-09-15T23:36:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7fadc948c640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc4162ab0 0x7fadc4201c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.425+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c0055500 con 0x7f18c0004a20 2024-09-15T23:36:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c004bfc0 con 0x7f18c0004a20 2024-09-15T23:36:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7fd373c9b640 1 -- 172.21.15.7:0/798043146 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd36c15fda0 con 0x7fd36c06edf0 2024-09-15T23:36:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f1a97fff640 1 -- 172.21.15.7:0/210916152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa800cca0 con 0x7f1ab00d3140 2024-09-15T23:36:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f1a97fff640 1 -- 172.21.15.7:0/210916152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8014680 con 0x7f1ab00d3140 2024-09-15T23:36:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7fd3627fc640 1 -- 172.21.15.7:0/798043146 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd368018020 con 0x7fd36c06edf0 2024-09-15T23:36:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7fd3627fc640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd35006c380 0x7fd35006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7f15177fe640 1 -- 172.21.15.7:0/3246347815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f151806cd40 msgr2=0x7f15180cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7f15177fe640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f151806cd40 0x7f15180cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7f15177fe640 1 -- 172.21.15.7:0/3246347815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151806e560 msgr2=0x7f1518200fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7f15177fe640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151806e560 0x7f1518200fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7f15177fe640 1 -- 172.21.15.7:0/3246347815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15180ce380 con 0x7f151806d120 2024-09-15T23:36:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7f15177fe640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151806d120 0x7f15180c8930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c390 tx=0x7f150c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f151ca8e640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151806e560 0x7f1518200fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7f1ab5fd6640 1 -- 172.21.15.7:0/210916152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab00b6b50 con 0x7f1ab00d3140 2024-09-15T23:36:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f18bdffb640 1 -- 172.21.15.7:0/164654938 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f18c8063410 con 0x7f18c0004a20 2024-09-15T23:36:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7fc80d7fa640 1 -- 172.21.15.7:0/917384607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800049070 con 0x7fc810154dd0 2024-09-15T23:36:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7fc80d7fa640 1 -- 172.21.15.7:0/917384607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc800004220 con 0x7fc810154dd0 2024-09-15T23:36:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8101f10f0 con 0x7fc810154dd0 2024-09-15T23:36:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7fadc9c8d640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc4072800 0x7fadc4159690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fadc0031000 tx=0x7fadc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7fadba7fc640 1 -- 172.21.15.7:0/2426920566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0037050 con 0x7fadc4072800 2024-09-15T23:36:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7fadba7fc640 1 -- 172.21.15.7:0/2426920566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc0038040 con 0x7fadc4072800 2024-09-15T23:36:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7fadba7fc640 1 -- 172.21.15.7:0/2426920566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0006e10 con 0x7fadc4072800 2024-09-15T23:36:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc4159e30 con 0x7fadc4072800 2024-09-15T23:36:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc415a3f0 con 0x7fadc4072800 2024-09-15T23:36:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7fc80d7fa640 1 -- 172.21.15.7:0/917384607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80003e040 con 0x7fc810154dd0 2024-09-15T23:36:45.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7f1ab5fd6640 1 -- 172.21.15.7:0/210916152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1ab0158460 con 0x7f1ab00d3140 2024-09-15T23:36:45.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8100af440 con 0x7fc810154dd0 2024-09-15T23:36:45.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f1514ff9640 1 -- 172.21.15.7:0/3246347815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c0074e0 con 0x7f151806d120 2024-09-15T23:36:45.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f1514ff9640 1 -- 172.21.15.7:0/3246347815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c0047e0 con 0x7f151806d120 2024-09-15T23:36:45.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f1514ff9640 1 -- 172.21.15.7:0/3246347815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c004bd0 con 0x7f151806d120 2024-09-15T23:36:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1518201a00 con 0x7f151806d120 2024-09-15T23:36:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7fd3627fc640 1 -- 172.21.15.7:0/798043146 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd36808b690 con 0x7fd36c06edf0 2024-09-15T23:36:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7f18bdffb640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f18ac06c380 0x7f18ac06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7fd3627fc640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7fd350075b00 0x7fd350077f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7fd371236640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd35006c380 0x7fd35006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7fd372238640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7fd350075b00 0x7fd350077f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7fd3627fc640 1 -- 172.21.15.7:0/798043146 --> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd368020ec0 con 0x7fd350075b00 2024-09-15T23:36:45.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7f18bdffb640 1 -- 172.21.15.7:0/164654938 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f18c8106dc0 con 0x7f18c0004a20 2024-09-15T23:36:45.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7f18ccbe7640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f18ac06c380 0x7f18ac06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fadc4159bd0 con 0x7fadc4072800 2024-09-15T23:36:45.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc81009a910 con 0x7fc810154dd0 2024-09-15T23:36:45.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f1a97fff640 1 -- 172.21.15.7:0/210916152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1aa801c070 con 0x7f1ab00d3140 2024-09-15T23:36:45.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.426+0000 7f31da01e640 1 Processor -- start 2024-09-15T23:36:45.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7f31da01e640 1 -- start start 2024-09-15T23:36:45.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f31da01e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 0x7f31cc0dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f31da01e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31cc036610 0x7f31cc036a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f18ccbe7640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f18ac06c380 0x7f18ac06e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18c4034000 tx=0x7f18c402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f31da01e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc036fd0 0x7f31cc034c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.427+0000 7ff54b7fe640 1 -- 172.21.15.7:0/1923736179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5540052d0 con 0x7ff5640b0530 2024-09-15T23:36:45.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7ff54b7fe640 1 -- 172.21.15.7:0/1923736179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554003830 con 0x7ff5640b0530 2024-09-15T23:36:45.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f31da01e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc007390 con 0x7f31c4004800 2024-09-15T23:36:45.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f31da01e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc0070b0 con 0x7f31cc036fd0 2024-09-15T23:36:45.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.428+0000 7f31da01e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f31cc007b90 con 0x7f31cc036610 2024-09-15T23:36:45.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7f31d37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 0x7f31cc0dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7fd3627fc640 1 -- 172.21.15.7:0/798043146 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd368092050 con 0x7fd36c06edf0 2024-09-15T23:36:45.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7fd372238640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7fd350075b00 0x7fd350077f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7fd371236640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd35006c380 0x7fd35006e840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd35c034000 tx=0x7fd35c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f18ce3ea640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7f18c004e670 0x7f18c00def90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 --> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18c004f2c0 con 0x7f18c004e670 2024-09-15T23:36:45.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7fc80d7fa640 1 -- 172.21.15.7:0/917384607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc8000399c0 con 0x7fc810154dd0 2024-09-15T23:36:45.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7f31d37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 0x7f31cc0dd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48306/0 (socket says 172.21.15.7:48306) 2024-09-15T23:36:45.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7f31d37fe640 1 -- 172.21.15.7:0/4091406130 learned_addr learned my addr 172.21.15.7:0/4091406130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.429+0000 7f31d2ffd640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31cc036610 0x7f31cc036a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f31d3fff640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc036fd0 0x7f31cc034c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f31d2ffd640 1 -- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc036fd0 msgr2=0x7f31cc034c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f31d2ffd640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc036fd0 0x7f31cc034c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f31d2ffd640 1 -- 172.21.15.7:0/4091406130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 msgr2=0x7f31cc0dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f31d2ffd640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31c4004800 0x7f31cc0dd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f31d2ffd640 1 -- 172.21.15.7:0/4091406130 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc037450 con 0x7f31cc036610 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f31d3fff640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc036fd0 0x7f31cc034c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f31d2ffd640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31cc036610 0x7f31cc036a90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f31c802f000 tx=0x7f31cc007670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.430+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15180c8130 con 0x7f151806d120 2024-09-15T23:36:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f31bffff640 1 -- 172.21.15.7:0/4091406130 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8037040 con 0x7f31cc036610 2024-09-15T23:36:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f18cdbe9640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7f18c004e670 0x7f18c00def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f31bffff640 1 -- 172.21.15.7:0/4091406130 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c8004a00 con 0x7f31cc036610 2024-09-15T23:36:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f1a97fff640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1a9c06c330 0x7f1a9c06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f1a97fff640 1 -- 172.21.15.7:0/210916152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1aa808b540 con 0x7f1ab00d3140 2024-09-15T23:36:45.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f31bffff640 1 -- 172.21.15.7:0/4091406130 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8007c60 con 0x7f31cc036610 2024-09-15T23:36:45.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f1a97fff640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f1a9c075a30 0x7f1a9c077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f1a97fff640 1 -- 172.21.15.7:0/210916152 --> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1aa800ce50 con 0x7f1a9c075a30 2024-09-15T23:36:45.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7fadba7fc640 1 -- 172.21.15.7:0/2426920566 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fadc0045020 con 0x7fadc4072800 2024-09-15T23:36:45.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7fc80d7fa640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc7fc06c380 0x7fc7fc06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f1aaffff640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f1a9c075a30 0x7f1a9c077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7f1a97fff640 1 -- 172.21.15.7:0/210916152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1aa8092050 con 0x7f1ab00d3140 2024-09-15T23:36:45.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7fc80d7fa640 1 -- 172.21.15.7:0/917384607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc8000b4520 con 0x7fc810154dd0 2024-09-15T23:36:45.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7fc80d7fa640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7fc7fc075a80 0x7fc7fc077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7fc80d7fa640 1 -- 172.21.15.7:0/917384607 --> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc800037710 con 0x7fc7fc075a80 2024-09-15T23:36:45.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7fc80d7fa640 1 -- 172.21.15.7:0/917384607 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc8000bb050 con 0x7fc810154dd0 2024-09-15T23:36:45.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31cc0dde20 con 0x7f31cc036610 2024-09-15T23:36:45.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.431+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f31cc04e0f0 con 0x7f31cc036610 2024-09-15T23:36:45.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1518201f20 con 0x7f151806d120 2024-09-15T23:36:45.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7fd3627fc640 1 -- 172.21.15.7:0/798043146 <== osd.0 v2:172.21.15.7:6819/1578659663 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd368020ec0 con 0x7fd350075b00 2024-09-15T23:36:45.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7fc814e17640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc7fc06c380 0x7fc7fc06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7f1aaf7fe640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1a9c06c330 0x7f1a9c06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7f1aaffff640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f1a9c075a30 0x7f1a9c077e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7fadba7fc640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fada806c380 0x7fada806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31cc04f1a0 con 0x7f31cc036610 2024-09-15T23:36:45.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7fadba7fc640 1 -- 172.21.15.7:0/2426920566 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fadc00ab670 con 0x7fadc4072800 2024-09-15T23:36:45.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.432+0000 7fc815618640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7fc7fc075a80 0x7fc7fc077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7f1aaf7fe640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1a9c06c330 0x7f1a9c06e7f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00095d0 tx=0x7f1aa0006b30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7fc815618640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7fc7fc075a80 0x7fc7fc077ec0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7fadba7fc640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7fada8075af0 0x7fada8077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7fadba7fc640 1 -- 172.21.15.7:0/2426920566 --> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fada8078600 con 0x7fada8075af0 2024-09-15T23:36:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7fadba7fc640 1 -- 172.21.15.7:0/2426920566 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadc00b2050 con 0x7fadc4072800 2024-09-15T23:36:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7ff54b7fe640 1 -- 172.21.15.7:0/1923736179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff55401a070 con 0x7ff5640b0530 2024-09-15T23:36:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7f18cdbe9640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7f18c004e670 0x7f18c00def90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7fadca48e640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7fada8075af0 0x7fada8077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7f31bffff640 1 -- 172.21.15.7:0/4091406130 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f31c8007070 con 0x7f31cc036610 2024-09-15T23:36:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7f1a97fff640 1 -- 172.21.15.7:0/210916152 <== osd.2 v2:172.21.15.7:6800/2860639344 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1aa800ce50 con 0x7f1a9c075a30 2024-09-15T23:36:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7fadc948c640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fada806c380 0x7fada806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.433+0000 7fc814e17640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc7fc06c380 0x7fc7fc06e840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc804006fd0 tx=0x7fc804008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7f18bdffb640 1 -- 172.21.15.7:0/164654938 <== osd.4 v2:172.21.15.18:6800/2251983885 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18c004f2c0 con 0x7f18c004e670 2024-09-15T23:36:45.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7fc80d7fa640 1 -- 172.21.15.7:0/917384607 <== osd.5 v2:172.21.15.18:6808/2875348132 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc800037710 con 0x7fc7fc075a80 2024-09-15T23:36:45.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7ff54b7fe640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff55006c380 0x7ff55006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7fadc948c640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fada806c380 0x7fada806e840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006fd0 tx=0x7fadb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7fadca48e640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7fada8075af0 0x7fada8077f30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7ff5627fc640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff55006c380 0x7ff55006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7f1514ff9640 1 -- 172.21.15.7:0/3246347815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f150c01a020 con 0x7f151806d120 2024-09-15T23:36:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.435+0000 7ff54b7fe640 1 -- 172.21.15.7:0/1923736179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff55408b1d0 con 0x7ff5640b0530 2024-09-15T23:36:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.434+0000 7f31bffff640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f31b806c380 0x7f31b806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.435+0000 7f31d37fe640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f31b806c380 0x7f31b806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.435+0000 7f1514ff9640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f150406c380 0x7f150406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.435+0000 7fadba7fc640 1 -- 172.21.15.7:0/2426920566 <== osd.6 v2:172.21.15.18:6824/468690220 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fada8078600 con 0x7fada8075af0 2024-09-15T23:36:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.435+0000 7ff5627fc640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff55006c380 0x7ff55006e840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff564154450 tx=0x7ff558009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.435+0000 7ff54b7fe640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff550075b00 0x7ff550077f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.435+0000 7ff54b7fe640 1 -- 172.21.15.7:0/1923736179 --> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5540052d0 con 0x7ff550075b00 2024-09-15T23:36:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7ff54b7fe640 1 -- 172.21.15.7:0/1923736179 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff55408b5c0 con 0x7ff5640b0530 2024-09-15T23:36:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7ff5637fe640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff550075b00 0x7ff550077f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.435+0000 7f31d37fe640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f31b806c380 0x7f31b806e840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31c0006fd0 tx=0x7f31c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f31bffff640 1 -- 172.21.15.7:0/4091406130 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f31c80abb70 con 0x7f31cc036610 2024-09-15T23:36:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f31bffff640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f31b8075a70 0x7f31b8077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f31bffff640 1 -- 172.21.15.7:0/4091406130 --> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31c800bf30 con 0x7f31b8075a70 2024-09-15T23:36:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f31bffff640 1 -- 172.21.15.7:0/4091406130 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31c80b2050 con 0x7f31cc036610 2024-09-15T23:36:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.435+0000 7f1517fff640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f150406c380 0x7f150406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f1514ff9640 1 -- 172.21.15.7:0/3246347815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f150c08b990 con 0x7f151806d120 2024-09-15T23:36:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f1517fff640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f150406c380 0x7f150406e840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1510009850 tx=0x7f1510009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f1514ff9640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f1504075b00 0x7f1504077f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f1514ff9640 1 -- 172.21.15.7:0/3246347815 --> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f150c008e50 con 0x7f1504075b00 2024-09-15T23:36:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f1514ff9640 1 -- 172.21.15.7:0/3246347815 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f150c08bd80 con 0x7f151806d120 2024-09-15T23:36:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f151ca8e640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f1504075b00 0x7f1504077f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.437+0000 7f151ca8e640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f1504075b00 0x7f1504077f40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.436+0000 7f31d3fff640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f31b8075a70 0x7f31b8077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.437+0000 7f31d3fff640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f31b8075a70 0x7f31b8077eb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.437+0000 7ff5637fe640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff550075b00 0x7ff550077f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.438+0000 7f31bffff640 1 -- 172.21.15.7:0/4091406130 <== osd.7 v2:172.21.15.18:6816/1769482769 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31c800bf30 con 0x7f31b8075a70 2024-09-15T23:36:45.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.438+0000 7f1514ff9640 1 -- 172.21.15.7:0/3246347815 <== osd.1 v2:172.21.15.7:6816/885929812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f150c008e50 con 0x7f1504075b00 2024-09-15T23:36:45.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.438+0000 7ff54b7fe640 1 -- 172.21.15.7:0/1923736179 <== osd.3 v2:172.21.15.7:6801/2160174572 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff5540052d0 con 0x7ff550075b00 2024-09-15T23:36:45.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.464+0000 7fd373c9b640 1 -- 172.21.15.7:0/798043146 --> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd36c06e8b0 con 0x7fd350075b00 2024-09-15T23:36:45.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.464+0000 7f1ab5fd6640 1 -- 172.21.15.7:0/210916152 --> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1ab00d4f60 con 0x7f1a9c075a30 2024-09-15T23:36:45.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.464+0000 7f1a97fff640 1 -- 172.21.15.7:0/210916152 <== osd.2 v2:172.21.15.7:6800/2860639344 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1ab00d4f60 con 0x7f1a9c075a30 2024-09-15T23:36:45.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.464+0000 7fd3627fc640 1 -- 172.21.15.7:0/798043146 <== osd.0 v2:172.21.15.7:6819/1578659663 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd36c06e8b0 con 0x7fd350075b00 2024-09-15T23:36:45.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.465+0000 7f1a95ffb640 1 -- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f1a9c075a30 msgr2=0x7f1a9c077e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.465+0000 7f1a95ffb640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f1a9c075a30 0x7f1a9c077e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.465+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 --> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc81015b9c0 con 0x7fc7fc075a80 2024-09-15T23:36:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f1a95ffb640 1 -- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1a9c06c330 msgr2=0x7f1a9c06e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f1a95ffb640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1a9c06c330 0x7f1a9c06e7f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00095d0 tx=0x7f1aa0006b30 comp rx=0 tx=0).stop 2024-09-15T23:36:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f1a95ffb640 1 -- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab00d3140 msgr2=0x7f1ab0203ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f1a95ffb640 1 --2- 172.21.15.7:0/210916152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab00d3140 0x7f1ab0203ec0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800bd90 tx=0x7f1aa8009ef0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f1aaffff640 1 -- 172.21.15.7:0/210916152 reap_dead start 2024-09-15T23:36:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.465+0000 7fd34ffff640 1 -- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7fd350075b00 msgr2=0x7fd350077f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.465+0000 7fd34ffff640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7fd350075b00 0x7fd350077f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.465+0000 7fd34ffff640 1 -- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd35006c380 msgr2=0x7fd35006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.465+0000 7fd34ffff640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd35006c380 0x7fd35006e840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd35c034000 tx=0x7fd35c02e000 comp rx=0 tx=0).stop 2024-09-15T23:36:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.465+0000 7fd34ffff640 1 -- 172.21.15.7:0/798043146 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c06edf0 msgr2=0x7fd36c15f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.465+0000 7fd34ffff640 1 --2- 172.21.15.7:0/798043146 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd36c06edf0 0x7fd36c15f860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b3e0 tx=0x7fd36800b8a0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7fd372238640 1 -- 172.21.15.7:0/798043146 reap_dead start 2024-09-15T23:36:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7fd34ffff640 1 -- 172.21.15.7:0/798043146 shutdown_connections 2024-09-15T23:36:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7fd34ffff640 1 -- 172.21.15.7:0/798043146 >> 172.21.15.7:0/798043146 conn(0x7fd36c0b1e60 msgr2=0x7fd36c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f1a95ffb640 1 -- 172.21.15.7:0/210916152 shutdown_connections 2024-09-15T23:36:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f1a95ffb640 1 -- 172.21.15.7:0/210916152 >> 172.21.15.7:0/210916152 conn(0x7f1ab008bfd0 msgr2=0x7f1ab00b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7fc80d7fa640 1 -- 172.21.15.7:0/917384607 <== osd.5 v2:172.21.15.18:6808/2875348132 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc81015b9c0 con 0x7fc7fc075a80 2024-09-15T23:36:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f1a95ffb640 1 -- 172.21.15.7:0/210916152 shutdown_connections 2024-09-15T23:36:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f1a95ffb640 1 -- 172.21.15.7:0/210916152 wait complete. 2024-09-15T23:36:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7fd34ffff640 1 -- 172.21.15.7:0/798043146 shutdown_connections 2024-09-15T23:36:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7fd34ffff640 1 -- 172.21.15.7:0/798043146 wait complete. 2024-09-15T23:36:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.466+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 --> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18c00d6a60 con 0x7f18c004e670 2024-09-15T23:36:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7fc7fc075a80 msgr2=0x7fc7fc077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc81707b640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7fc7fc075a80 0x7fc7fc077ec0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc7fc06c380 msgr2=0x7fc7fc06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc81707b640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc7fc06c380 0x7fc7fc06e840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc804006fd0 tx=0x7fc804008040 comp rx=0 tx=0).stop 2024-09-15T23:36:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc810154dd0 msgr2=0x7fc8101ed710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc81707b640 1 --2- 172.21.15.7:0/917384607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc810154dd0 0x7fc8101ed710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc8100d1220 tx=0x7fc8000046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc815618640 1 -- 172.21.15.7:0/917384607 reap_dead start 2024-09-15T23:36:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 shutdown_connections 2024-09-15T23:36:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 >> 172.21.15.7:0/917384607 conn(0x7fc810072430 msgr2=0x7fc8100b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7f18bdffb640 1 -- 172.21.15.7:0/164654938 <== osd.4 v2:172.21.15.18:6800/2251983885 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f18c00d6a60 con 0x7f18c004e670 2024-09-15T23:36:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 --> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fadc40cfef0 con 0x7fada8075af0 2024-09-15T23:36:45.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.467+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 shutdown_connections 2024-09-15T23:36:45.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7fc81707b640 1 -- 172.21.15.7:0/917384607 wait complete. 2024-09-15T23:36:45.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7fadba7fc640 1 -- 172.21.15.7:0/2426920566 <== osd.6 v2:172.21.15.18:6824/468690220 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fadc40cfef0 con 0x7fada8075af0 2024-09-15T23:36:45.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7fada8075af0 msgr2=0x7fada8077f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7fadcbef1640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7fada8075af0 0x7fada8077f30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 --> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f31cc052e70 con 0x7f31b8075a70 2024-09-15T23:36:45.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fada806c380 msgr2=0x7fada806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7fadcbef1640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fada806c380 0x7fada806e840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006fd0 tx=0x7fadb4008040 comp rx=0 tx=0).stop 2024-09-15T23:36:45.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc4072800 msgr2=0x7fadc4159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7fadcbef1640 1 --2- 172.21.15.7:0/2426920566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc4072800 0x7fadc4159690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fadc0031000 tx=0x7fadc00046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.468+0000 7fadca48e640 1 -- 172.21.15.7:0/2426920566 reap_dead start 2024-09-15T23:36:45.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7f18c004e670 msgr2=0x7f18c00def90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18ce3ea640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7f18c004e670 0x7f18c00def90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 shutdown_connections 2024-09-15T23:36:45.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 >> 172.21.15.7:0/2426920566 conn(0x7fadc40b25d0 msgr2=0x7fadc41576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f31bffff640 1 -- 172.21.15.7:0/4091406130 <== osd.7 v2:172.21.15.18:6816/1769482769 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f31cc052e70 con 0x7f31b8075a70 2024-09-15T23:36:45.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 shutdown_connections 2024-09-15T23:36:45.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f18ac06c380 msgr2=0x7f18ac06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7fadcbef1640 1 -- 172.21.15.7:0/2426920566 wait complete. 2024-09-15T23:36:45.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18ce3ea640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f18ac06c380 0x7f18ac06e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18c4034000 tx=0x7f18c402e000 comp rx=0 tx=0).stop 2024-09-15T23:36:45.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f18c0004a20 msgr2=0x7f18c0188540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18ce3ea640 1 --2- 172.21.15.7:0/164654938 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f18c0004a20 0x7f18c0188540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f18c802dd50 tx=0x7f18c8069e70 comp rx=0 tx=0).stop 2024-09-15T23:36:45.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 --> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1518201f20 con 0x7f1504075b00 2024-09-15T23:36:45.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18cdbe9640 1 -- 172.21.15.7:0/164654938 reap_dead start 2024-09-15T23:36:45.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f31b8075a70 msgr2=0x7f31b8077eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f31da01e640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f31b8075a70 0x7f31b8077eb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 shutdown_connections 2024-09-15T23:36:45.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 >> 172.21.15.7:0/164654938 conn(0x7f18c001dd70 msgr2=0x7f18c0032220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.469+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 shutdown_connections 2024-09-15T23:36:45.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f18ce3ea640 1 -- 172.21.15.7:0/164654938 wait complete. 2024-09-15T23:36:45.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 --> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff5640b6620 con 0x7ff550075b00 2024-09-15T23:36:45.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f31b806c380 msgr2=0x7f31b806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f31da01e640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f31b806c380 0x7f31b806e840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31c0006fd0 tx=0x7f31c0008040 comp rx=0 tx=0).stop 2024-09-15T23:36:45.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31cc036610 msgr2=0x7f31cc036a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f31da01e640 1 --2- 172.21.15.7:0/4091406130 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31cc036610 0x7f31cc036a90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f31c802f000 tx=0x7f31cc007670 comp rx=0 tx=0).stop 2024-09-15T23:36:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f1514ff9640 1 -- 172.21.15.7:0/3246347815 <== osd.1 v2:172.21.15.7:6816/885929812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1508009840 con 0x7f1504075b00 2024-09-15T23:36:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f31d3fff640 1 -- 172.21.15.7:0/4091406130 reap_dead start 2024-09-15T23:36:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 shutdown_connections 2024-09-15T23:36:45.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.470+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 >> 172.21.15.7:0/4091406130 conn(0x7f31cc01e1c0 msgr2=0x7f31cc050200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f1504075b00 msgr2=0x7f1504077f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7f151e4f1640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f1504075b00 0x7f1504077f40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f150406c380 msgr2=0x7f150406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7ff54b7fe640 1 -- 172.21.15.7:0/1923736179 <== osd.3 v2:172.21.15.7:6801/2160174572 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff5640b6620 con 0x7ff550075b00 2024-09-15T23:36:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7f151e4f1640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f150406c380 0x7f150406e840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1510009850 tx=0x7f1510009000 comp rx=0 tx=0).stop 2024-09-15T23:36:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 shutdown_connections 2024-09-15T23:36:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7f31da01e640 1 -- 172.21.15.7:0/4091406130 wait complete. 2024-09-15T23:36:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151806d120 msgr2=0x7f15180c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7f151e4f1640 1 --2- 172.21.15.7:0/3246347815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151806d120 0x7f15180c8930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c390 tx=0x7f150c00c850 comp rx=0 tx=0).stop 2024-09-15T23:36:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.471+0000 7f151ca8e640 1 -- 172.21.15.7:0/3246347815 reap_dead start 2024-09-15T23:36:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.472+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff550075b00 msgr2=0x7ff550077f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.472+0000 7ff5697d8640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff550075b00 0x7ff550077f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.472+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 shutdown_connections 2024-09-15T23:36:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.472+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 >> 172.21.15.7:0/3246347815 conn(0x7f151808bfd0 msgr2=0x7f15180b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.472+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff55006c380 msgr2=0x7ff55006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.472+0000 7ff5697d8640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff55006c380 0x7ff55006e840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff564154450 tx=0x7ff558009040 comp rx=0 tx=0).stop 2024-09-15T23:36:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.472+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 msgr2=0x7ff5640ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.472+0000 7ff5697d8640 1 --2- 172.21.15.7:0/1923736179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b0530 0x7ff5640ceff0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c400 tx=0x7ff55400c8c0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.472+0000 7ff5637fe640 1 -- 172.21.15.7:0/1923736179 reap_dead start 2024-09-15T23:36:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.473+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 shutdown_connections 2024-09-15T23:36:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.473+0000 7f151e4f1640 1 -- 172.21.15.7:0/3246347815 wait complete. 2024-09-15T23:36:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.473+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 shutdown_connections 2024-09-15T23:36:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.473+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 >> 172.21.15.7:0/1923736179 conn(0x7ff5640734b0 msgr2=0x7ff564161230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.473+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 shutdown_connections 2024-09-15T23:36:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.473+0000 7ff5697d8640 1 -- 172.21.15.7:0/1923736179 wait complete. 2024-09-15T23:36:45.563 INFO:teuthology.orchestra.run.smithi007.stdout:64424509449 2024-09-15T23:36:45.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T23:36:45.564 INFO:teuthology.orchestra.run.smithi007.stdout:64424509450 2024-09-15T23:36:45.564 INFO:teuthology.orchestra.run.smithi007.stdout:55834574858 2024-09-15T23:36:45.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T23:36:45.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T23:36:45.566 INFO:teuthology.orchestra.run.smithi007.stdout:55834574857 2024-09-15T23:36:45.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T23:36:45.567 INFO:teuthology.orchestra.run.smithi007.stdout:60129542152 2024-09-15T23:36:45.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T23:36:45.568 INFO:teuthology.orchestra.run.smithi007.stdout:60129542152 2024-09-15T23:36:45.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T23:36:45.569 INFO:teuthology.orchestra.run.smithi007.stdout:64424509449 2024-09-15T23:36:45.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T23:36:45.570 INFO:teuthology.orchestra.run.smithi007.stdout:64424509449 2024-09-15T23:36:45.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T23:36:45.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.781+0000 7f60cf36d640 1 Processor -- start 2024-09-15T23:36:45.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.781+0000 7f60cf36d640 1 -- start start 2024-09-15T23:36:45.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.781+0000 7f8de2043640 1 Processor -- start 2024-09-15T23:36:45.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.781+0000 7f8de2043640 1 -- start start 2024-09-15T23:36:45.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.782+0000 7f577e396640 1 Processor -- start 2024-09-15T23:36:45.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f8029c74640 1 Processor -- start 2024-09-15T23:36:45.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f8029c74640 1 -- start start 2024-09-15T23:36:45.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f577e396640 1 -- start start 2024-09-15T23:36:45.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cf36d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60c80b2570 0x7f60c815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cf36d640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f60c80b3b70 0x7f60c80b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cf36d640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c80b44c0 con 0x7f60c8154dd0 2024-09-15T23:36:45.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cf36d640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f60c80732e0 con 0x7f60c80b2570 2024-09-15T23:36:45.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cf36d640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f60c8073e60 con 0x7f60c80b3b70 2024-09-15T23:36:45.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cd109640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60c80b2570 0x7f60c815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cd109640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60c80b2570 0x7f60c815b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48320/0 (socket says 172.21.15.7:48320) 2024-09-15T23:36:45.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cd109640 1 -- 172.21.15.7:0/2224053941 learned_addr learned my addr 172.21.15.7:0/2224053941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f577e396640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f577806f1c0 0x7f577815a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f577e396640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f577815ac50 con 0x7f57780b5be0 2024-09-15T23:36:45.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f577e396640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f577806f7a0 con 0x7f577806cc20 2024-09-15T23:36:45.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f577e396640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5778068060 con 0x7f577806f1c0 2024-09-15T23:36:45.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8029c74640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 0x7f801c006670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8029c74640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f801c034360 0x7f801c054820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8029c74640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f801c0348b0 0x7f801c054d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8029c74640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f801c0089c0 con 0x7f801c034360 2024-09-15T23:36:45.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8029c74640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f801c008ef0 con 0x7f801c0348b0 2024-09-15T23:36:45.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8029c74640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f801c033160 con 0x7f801c0e2e90 2024-09-15T23:36:45.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7fbd39065640 1 Processor -- start 2024-09-15T23:36:45.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cd109640 1 -- 172.21.15.7:0/2224053941 >> v2:172.21.15.18:3300/0 conn(0x7f60c80b3b70 msgr2=0x7f60c80b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cd109640 1 --2- 172.21.15.7:0/2224053941 >> v2:172.21.15.18:3300/0 conn(0x7f60c80b3b70 0x7f60c80b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cd109640 1 -- 172.21.15.7:0/2224053941 >> v1:172.21.15.18:6789/0 conn(0x7f60c8154dd0 legacy=0x7f60c80b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f60cd109640 1 -- 172.21.15.7:0/2224053941 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c8155260 con 0x7f60c80b2570 2024-09-15T23:36:45.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f8023fff640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f801c0348b0 0x7f801c054d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f8023fff640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f801c0348b0 0x7f801c054d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:47258/0 (socket says 172.21.15.7:47258) 2024-09-15T23:36:45.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f8023fff640 1 -- 172.21.15.7:0/205148805 learned_addr learned my addr 172.21.15.7:0/205148805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7fbd39065640 1 -- start start 2024-09-15T23:36:45.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7fce9ef52640 1 Processor -- start 2024-09-15T23:36:45.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f62121e0640 1 Processor -- start 2024-09-15T23:36:45.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f62121e0640 1 -- start start 2024-09-15T23:36:45.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f60cd109640 1 --2- 172.21.15.7:0/2224053941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 0x7f60c815b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f60c8073460 tx=0x7f60c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6978e31559271780 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.782+0000 7f8de2043640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8dd4054580 0x7f8dd40514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.782+0000 7f8de2043640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd4051a10 0x7f8dd4051e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.782+0000 7f8de2043640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd400bff0 con 0x7f8dd4054d80 2024-09-15T23:36:45.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f8de2043640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd400bda0 con 0x7f8dd4051a10 2024-09-15T23:36:45.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f8de2043640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd400c2d0 con 0x7f8dd4054580 2024-09-15T23:36:45.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f8de0840640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8dd4054580 0x7f8dd40514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.783+0000 7f8de1842640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd4051a10 0x7f8dd4051e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8de1842640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd4051a10 0x7f8dd4051e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48312/0 (socket says 172.21.15.7:48312) 2024-09-15T23:36:45.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f62121e0640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f62040e7040 0x7f6204006490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f62121e0640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62040538c0 0x7f6204053ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8de1842640 1 -- 172.21.15.7:0/939403593 learned_addr learned my addr 172.21.15.7:0/939403593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8de1842640 1 -- 172.21.15.7:0/939403593 >> v2:172.21.15.18:3300/0 conn(0x7f8dd4054580 msgr2=0x7f8dd40514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8de1842640 1 --2- 172.21.15.7:0/939403593 >> v2:172.21.15.18:3300/0 conn(0x7f8dd4054580 0x7f8dd40514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8de1842640 1 -- 172.21.15.7:0/939403593 >> v1:172.21.15.7:6789/0 conn(0x7f8dd4054d80 legacy=0x7f8dd4055160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:36:45.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.784+0000 7f8de1842640 1 -- 172.21.15.7:0/939403593 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd400c5b0 con 0x7f8dd4051a10 2024-09-15T23:36:45.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f0093227640 1 Processor -- start 2024-09-15T23:36:45.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7fce9ef52640 1 -- start start 2024-09-15T23:36:45.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f62121e0640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62040e7420 con 0x7f62040eab20 2024-09-15T23:36:45.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f62121e0640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6204032870 con 0x7f62040538c0 2024-09-15T23:36:45.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f62121e0640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f6204047d80 con 0x7f62040e7040 2024-09-15T23:36:45.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f62111de640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f62040e7040 0x7f6204006490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f577d394640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f577806f1c0 0x7f577815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f577db95640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f577806cc20 0x7f57780cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:35454/0 (socket says 172.21.15.7:35454) 2024-09-15T23:36:45.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f577db95640 1 -- 172.21.15.7:0/2018620062 learned_addr learned my addr 172.21.15.7:0/2018620062 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:45.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f577d394640 1 -- 172.21.15.7:0/2018620062 >> v1:172.21.15.18:6789/0 conn(0x7f577806cc20 legacy=0x7f57780cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f577d394640 1 -- 172.21.15.7:0/2018620062 >> v1:172.21.15.7:6789/0 conn(0x7f57780b5be0 legacy=0x7f5778067950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f577d394640 1 -- 172.21.15.7:0/2018620062 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57781612d0 con 0x7f577806f1c0 2024-09-15T23:36:45.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f62111de640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f62040e7040 0x7f6204006490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:47274/0 (socket says 172.21.15.7:47274) 2024-09-15T23:36:45.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f62111de640 1 -- 172.21.15.7:0/1914753635 learned_addr learned my addr 172.21.15.7:0/1914753635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7fbd39065640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd34072600 0x7fbd3406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7fbd39065640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd3406f570 con 0x7fbd34165fa0 2024-09-15T23:36:45.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7fbd39065640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd3406f850 con 0x7fbd3406edf0 2024-09-15T23:36:45.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7fbd39065640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbd34166380 con 0x7fbd34072600 2024-09-15T23:36:45.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7fbd3259c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd34072600 0x7fbd3406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7fbd3259c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd34072600 0x7fbd3406e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48354/0 (socket says 172.21.15.7:48354) 2024-09-15T23:36:45.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7fbd3259c640 1 -- 172.21.15.7:0/2188250234 learned_addr learned my addr 172.21.15.7:0/2188250234 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f62111de640 1 -- 172.21.15.7:0/1914753635 >> v2:172.21.15.7:3300/0 conn(0x7f62040538c0 msgr2=0x7f6204053ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f62109dd640 1 --2- 172.21.15.7:0/1914753635 >> v2:172.21.15.7:3300/0 conn(0x7f62040538c0 0x7f6204053ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f62111de640 1 --2- 172.21.15.7:0/1914753635 >> v2:172.21.15.7:3300/0 conn(0x7f62040538c0 0x7f6204053ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f62111de640 1 -- 172.21.15.7:0/1914753635 >> v1:172.21.15.7:6789/0 conn(0x7f62040eab20 legacy=0x7f62040eaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f62111de640 1 -- 172.21.15.7:0/1914753635 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6204006e10 con 0x7f62040e7040 2024-09-15T23:36:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f62109dd640 1 --2- 172.21.15.7:0/1914753635 >> v2:172.21.15.7:3300/0 conn(0x7f62040538c0 0x7f6204053ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f577d394640 1 --2- 172.21.15.7:0/2018620062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577806f1c0 0x7f577815a710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5774009720 tx=0x7f577402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9a464ec0958f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f8de1842640 1 --2- 172.21.15.7:0/939403593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4051a10 0x7f8dd4051e40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4036190 tx=0x7f8dd803b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aa874ba489ca95 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f0093227640 1 -- start start 2024-09-15T23:36:45.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f0093227640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c154dd0 0x7f008c15b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f0093227640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f008c0b3160 0x7f008c0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f0093227640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f008c0b40b0 0x7f008c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f0093227640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f008c0732e0 con 0x7f008c0b3160 2024-09-15T23:36:45.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f0093227640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f008c073e60 con 0x7f008c0b40b0 2024-09-15T23:36:45.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f0093227640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f008c073530 con 0x7f008c154dd0 2024-09-15T23:36:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fce9ef52640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce98154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fce9ef52640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce98073810 con 0x7fce9815b9c0 2024-09-15T23:36:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fce9ef52640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce98073af0 con 0x7fce980b2570 2024-09-15T23:36:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fce9ef52640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fce980732e0 con 0x7fce980b4280 2024-09-15T23:36:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f80237fe640 1 --2- 172.21.15.7:0/205148805 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 0x7f801c006670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.785+0000 7f8022ffd640 1 --2- 172.21.15.7:0/205148805 >> v2:172.21.15.7:3300/0 conn(0x7f801c034360 0x7f801c054820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f8023fff640 1 -- 172.21.15.7:0/205148805 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 msgr2=0x7f801c006670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f8023fff640 1 --2- 172.21.15.7:0/205148805 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 0x7f801c006670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fbd3259c640 1 -- 172.21.15.7:0/2188250234 >> v1:172.21.15.18:6789/0 conn(0x7fbd3406edf0 legacy=0x7fbd340b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fbd3259c640 1 -- 172.21.15.7:0/2188250234 >> v1:172.21.15.7:6789/0 conn(0x7fbd34165fa0 legacy=0x7fbd34071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fbd3259c640 1 -- 172.21.15.7:0/2188250234 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd340b39d0 con 0x7fbd34072600 2024-09-15T23:36:45.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f8023fff640 1 -- 172.21.15.7:0/205148805 >> v2:172.21.15.7:3300/0 conn(0x7f801c034360 msgr2=0x7f801c054820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f8023fff640 1 --2- 172.21.15.7:0/205148805 >> v2:172.21.15.7:3300/0 conn(0x7f801c034360 0x7f801c054820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f8023fff640 1 -- 172.21.15.7:0/205148805 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c008c10 con 0x7f801c0348b0 2024-09-15T23:36:45.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.786+0000 7f80237fe640 1 --2- 172.21.15.7:0/205148805 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 0x7f801c006670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f8dd3fff640 1 -- 172.21.15.7:0/939403593 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80110d0 con 0x7f8dd4051a10 2024-09-15T23:36:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f00917c4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f008c0b40b0 0x7f008c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f00917c4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f008c0b40b0 0x7f008c15e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:47290/0 (socket says 172.21.15.7:47290) 2024-09-15T23:36:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f008bfff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f008c0b3160 0x7f008c0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f8dd3fff640 1 -- 172.21.15.7:0/939403593 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd8011270 con 0x7f8dd4051a10 2024-09-15T23:36:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f8dd3fff640 1 -- 172.21.15.7:0/939403593 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8011640 con 0x7f8dd4051a10 2024-09-15T23:36:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f00917c4640 1 -- 172.21.15.7:0/466337334 learned_addr learned my addr 172.21.15.7:0/466337334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fce9d4ef640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce98154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fce9d4ef640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce98154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:32948/0 (socket says 172.21.15.7:32948) 2024-09-15T23:36:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7fce9d4ef640 1 -- 172.21.15.7:0/914442151 learned_addr learned my addr 172.21.15.7:0/914442151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f60bffff640 1 -- 172.21.15.7:0/2224053941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4037070 con 0x7f60c80b2570 2024-09-15T23:36:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f60bffff640 1 -- 172.21.15.7:0/2224053941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c4032440 con 0x7f60c80b2570 2024-09-15T23:36:45.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f60bffff640 1 -- 172.21.15.7:0/2224053941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40327f0 con 0x7f60c80b2570 2024-09-15T23:36:45.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f8023fff640 1 --2- 172.21.15.7:0/205148805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 0x7f801c054d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f801c039600 tx=0x7f801002a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a233871ec10c5de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.787+0000 7f0090fc3640 1 --2- 172.21.15.7:0/466337334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c154dd0 0x7f008c15b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f00917c4640 1 -- 172.21.15.7:0/466337334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c154dd0 msgr2=0x7f008c15b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7fce9d4ef640 1 -- 172.21.15.7:0/914442151 >> v1:172.21.15.18:6789/0 conn(0x7fce980b2570 legacy=0x7fce980b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7fce9d4ef640 1 -- 172.21.15.7:0/914442151 >> v1:172.21.15.7:6789/0 conn(0x7fce9815b9c0 legacy=0x7fce980b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7fce9d4ef640 1 -- 172.21.15.7:0/914442151 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce981614f0 con 0x7fce980b4280 2024-09-15T23:36:45.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f80227fc640 1 -- 172.21.15.7:0/205148805 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010037070 con 0x7f801c0348b0 2024-09-15T23:36:45.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f00917c4640 1 --2- 172.21.15.7:0/466337334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c154dd0 0x7f008c15b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f00917c4640 1 -- 172.21.15.7:0/466337334 >> v2:172.21.15.7:3300/0 conn(0x7f008c0b3160 msgr2=0x7f008c0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:45.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f00917c4640 1 --2- 172.21.15.7:0/466337334 >> v2:172.21.15.7:3300/0 conn(0x7f008c0b3160 0x7f008c0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f00917c4640 1 -- 172.21.15.7:0/466337334 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c073810 con 0x7f008c0b40b0 2024-09-15T23:36:45.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f80227fc640 1 -- 172.21.15.7:0/205148805 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8010032440 con 0x7f801c0348b0 2024-09-15T23:36:45.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f80227fc640 1 -- 172.21.15.7:0/205148805 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010032810 con 0x7f801c0348b0 2024-09-15T23:36:45.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7fbd3259c640 1 --2- 172.21.15.7:0/2188250234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd34072600 0x7fbd3406e8b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009e50 tx=0x7fbd2c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a86689dd29565a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f576ffff640 1 -- 172.21.15.7:0/2018620062 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774037070 con 0x7f577806f1c0 2024-09-15T23:36:45.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f62111de640 1 --2- 172.21.15.7:0/1914753635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62040e7040 0x7f6204006490 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6204008070 tx=0x7f620802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3047443a6c86ac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f60cf36d640 1 -- 172.21.15.7:0/2224053941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 msgr2=0x7f60c815b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7fbd31d9b640 1 -- 172.21.15.7:0/2188250234 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c036070 con 0x7fbd34072600 2024-09-15T23:36:45.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f6203fff640 1 -- 172.21.15.7:0/1914753635 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208004030 con 0x7f62040e7040 2024-09-15T23:36:45.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f60cf36d640 1 --2- 172.21.15.7:0/2224053941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 0x7f60c815b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f60c8073460 tx=0x7f60c402a230 comp rx=0 tx=0).stop 2024-09-15T23:36:45.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7fbd31d9b640 1 -- 172.21.15.7:0/2188250234 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c031600 con 0x7fbd34072600 2024-09-15T23:36:45.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f576ffff640 1 -- 172.21.15.7:0/2018620062 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577402ad70 con 0x7f577806f1c0 2024-09-15T23:36:45.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7fbd31d9b640 1 -- 172.21.15.7:0/2188250234 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c036070 con 0x7fbd34072600 2024-09-15T23:36:45.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8de2043640 1 -- 172.21.15.7:0/939403593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4051a10 msgr2=0x7f8dd4051e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8de2043640 1 --2- 172.21.15.7:0/939403593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4051a10 0x7f8dd4051e40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4036190 tx=0x7f8dd803b390 comp rx=0 tx=0).stop 2024-09-15T23:36:45.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f60cf36d640 1 -- 172.21.15.7:0/2224053941 shutdown_connections 2024-09-15T23:36:45.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f60cf36d640 1 --2- 172.21.15.7:0/2224053941 >> v2:172.21.15.18:3300/0 conn(0x7f60c80b3b70 0x7f60c80b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f60cf36d640 1 --2- 172.21.15.7:0/2224053941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 0x7f60c815b9c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f60cf36d640 1 -- 172.21.15.7:0/2224053941 >> 172.21.15.7:0/2224053941 conn(0x7f60c8072430 msgr2=0x7f60c80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7fce9d4ef640 1 --2- 172.21.15.7:0/914442151 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce98154dd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fce90009e80 tx=0x7fce9002a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d43fb0adf36823 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.788+0000 7f6203fff640 1 -- 172.21.15.7:0/1914753635 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62080041d0 con 0x7f62040e7040 2024-09-15T23:36:45.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f6203fff640 1 -- 172.21.15.7:0/1914753635 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208004030 con 0x7f62040e7040 2024-09-15T23:36:45.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f60cf36d640 1 -- 172.21.15.7:0/2224053941 shutdown_connections 2024-09-15T23:36:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f008bfff640 1 --2- 172.21.15.7:0/466337334 >> v2:172.21.15.7:3300/0 conn(0x7f008c0b3160 0x7f008c0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f0090fc3640 1 --2- 172.21.15.7:0/466337334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c154dd0 0x7f008c15b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8de2043640 1 -- 172.21.15.7:0/939403593 shutdown_connections 2024-09-15T23:36:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8de2043640 1 --2- 172.21.15.7:0/939403593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4051a10 0x7f8dd4051e40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8de2043640 1 --2- 172.21.15.7:0/939403593 >> v2:172.21.15.18:3300/0 conn(0x7f8dd4054580 0x7f8dd40514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8de2043640 1 -- 172.21.15.7:0/939403593 >> 172.21.15.7:0/939403593 conn(0x7f8dd40567a0 msgr2=0x7f8dd4056bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7fce977fe640 1 -- 172.21.15.7:0/914442151 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90036070 con 0x7fce980b4280 2024-09-15T23:36:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7fce977fe640 1 -- 172.21.15.7:0/914442151 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce90031600 con 0x7fce980b4280 2024-09-15T23:36:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8029c74640 1 -- 172.21.15.7:0/205148805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 msgr2=0x7f801c054d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8029c74640 1 --2- 172.21.15.7:0/205148805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 0x7f801c054d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f801c039600 tx=0x7f801002a160 comp rx=0 tx=0).stop 2024-09-15T23:36:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8029c74640 1 -- 172.21.15.7:0/205148805 shutdown_connections 2024-09-15T23:36:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8029c74640 1 --2- 172.21.15.7:0/205148805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 0x7f801c054d60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8029c74640 1 --2- 172.21.15.7:0/205148805 >> v2:172.21.15.7:3300/0 conn(0x7f801c034360 0x7f801c054820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8029c74640 1 --2- 172.21.15.7:0/205148805 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 0x7f801c006670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8029c74640 1 -- 172.21.15.7:0/205148805 >> 172.21.15.7:0/205148805 conn(0x7f801c01ea50 msgr2=0x7f801c038650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7fce977fe640 1 -- 172.21.15.7:0/914442151 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce900319d0 con 0x7fce980b4280 2024-09-15T23:36:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f576ffff640 1 -- 172.21.15.7:0/2018620062 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774032660 con 0x7f577806f1c0 2024-09-15T23:36:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8de2043640 1 -- 172.21.15.7:0/939403593 shutdown_connections 2024-09-15T23:36:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7fbd39065640 1 -- 172.21.15.7:0/2188250234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd34072600 msgr2=0x7fbd3406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7fbd39065640 1 --2- 172.21.15.7:0/2188250234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd34072600 0x7fbd3406e8b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009e50 tx=0x7fbd2c02a7c0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f62121e0640 1 -- 172.21.15.7:0/1914753635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62040e7040 msgr2=0x7f6204006490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f62121e0640 1 --2- 172.21.15.7:0/1914753635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62040e7040 0x7f6204006490 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6204008070 tx=0x7f620802a210 comp rx=0 tx=0).stop 2024-09-15T23:36:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f60cf36d640 1 -- 172.21.15.7:0/2224053941 wait complete. 2024-09-15T23:36:45.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.789+0000 7f8029c74640 1 -- 172.21.15.7:0/205148805 shutdown_connections 2024-09-15T23:36:45.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fbd39065640 1 -- 172.21.15.7:0/2188250234 shutdown_connections 2024-09-15T23:36:45.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f62121e0640 1 -- 172.21.15.7:0/1914753635 shutdown_connections 2024-09-15T23:36:45.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f62121e0640 1 --2- 172.21.15.7:0/1914753635 >> v2:172.21.15.7:3300/0 conn(0x7f62040538c0 0x7f6204053ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f62121e0640 1 --2- 172.21.15.7:0/1914753635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62040e7040 0x7f6204006490 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f62121e0640 1 -- 172.21.15.7:0/1914753635 >> 172.21.15.7:0/1914753635 conn(0x7f6204036090 msgr2=0x7f62040364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f8029c74640 1 -- 172.21.15.7:0/205148805 wait complete. 2024-09-15T23:36:45.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fbd39065640 1 --2- 172.21.15.7:0/2188250234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd34072600 0x7fbd3406e8b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f00917c4640 1 --2- 172.21.15.7:0/466337334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c0b40b0 0x7f008c15e3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f008c15f440 tx=0x7f008402a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c7c539c7ab9f1ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fbd39065640 1 -- 172.21.15.7:0/2188250234 >> 172.21.15.7:0/2188250234 conn(0x7fbd340b1e60 msgr2=0x7fbd340b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f62121e0640 1 -- 172.21.15.7:0/1914753635 shutdown_connections 2024-09-15T23:36:45.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fbd39065640 1 -- 172.21.15.7:0/2188250234 shutdown_connections 2024-09-15T23:36:45.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fce9ef52640 1 -- 172.21.15.7:0/914442151 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 msgr2=0x7fce98154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fce9ef52640 1 --2- 172.21.15.7:0/914442151 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce98154dd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fce90009e80 tx=0x7fce9002a7a0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f8de2043640 1 -- 172.21.15.7:0/939403593 wait complete. 2024-09-15T23:36:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fce9ef52640 1 -- 172.21.15.7:0/914442151 shutdown_connections 2024-09-15T23:36:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fce9ef52640 1 --2- 172.21.15.7:0/914442151 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce98154dd0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fce9ef52640 1 -- 172.21.15.7:0/914442151 >> 172.21.15.7:0/914442151 conn(0x7fce98072430 msgr2=0x7fce980b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fce9ef52640 1 -- 172.21.15.7:0/914442151 shutdown_connections 2024-09-15T23:36:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fce9ef52640 1 -- 172.21.15.7:0/914442151 wait complete. 2024-09-15T23:36:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f577e396640 1 -- 172.21.15.7:0/2018620062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577806f1c0 msgr2=0x7f577815a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f577e396640 1 --2- 172.21.15.7:0/2018620062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577806f1c0 0x7f577815a710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5774009720 tx=0x7f577402a060 comp rx=0 tx=0).stop 2024-09-15T23:36:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f008b7fe640 1 -- 172.21.15.7:0/466337334 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084036070 con 0x7f008c0b40b0 2024-09-15T23:36:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f008b7fe640 1 -- 172.21.15.7:0/466337334 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00840090d0 con 0x7f008c0b40b0 2024-09-15T23:36:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f008b7fe640 1 -- 172.21.15.7:0/466337334 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00840094a0 con 0x7f008c0b40b0 2024-09-15T23:36:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f60cf36d640 1 Processor -- start 2024-09-15T23:36:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f577e396640 1 -- 172.21.15.7:0/2018620062 shutdown_connections 2024-09-15T23:36:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f577e396640 1 --2- 172.21.15.7:0/2018620062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577806f1c0 0x7f577815a710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f577e396640 1 -- 172.21.15.7:0/2018620062 >> 172.21.15.7:0/2018620062 conn(0x7f5778072640 msgr2=0x7f5778072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f60cf36d640 1 -- start start 2024-09-15T23:36:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f577e396640 1 -- 172.21.15.7:0/2018620062 shutdown_connections 2024-09-15T23:36:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f577e396640 1 -- 172.21.15.7:0/2018620062 wait complete. 2024-09-15T23:36:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7f8de2043640 1 Processor -- start 2024-09-15T23:36:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7fbd39065640 1 -- 172.21.15.7:0/2188250234 wait complete. 2024-09-15T23:36:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7f0093227640 1 -- 172.21.15.7:0/466337334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c0b40b0 msgr2=0x7f008c15e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7f8de2043640 1 -- start start 2024-09-15T23:36:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7f0093227640 1 --2- 172.21.15.7:0/466337334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c0b40b0 0x7f008c15e3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f008c15f440 tx=0x7f008402a620 comp rx=0 tx=0).stop 2024-09-15T23:36:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7fce9ef52640 1 Processor -- start 2024-09-15T23:36:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7fce9ef52640 1 -- start start 2024-09-15T23:36:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.791+0000 7f8029c74640 1 Processor -- start 2024-09-15T23:36:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7f8029c74640 1 -- start start 2024-09-15T23:36:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7fbd39065640 1 Processor -- start 2024-09-15T23:36:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7fbd39065640 1 -- start start 2024-09-15T23:36:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.790+0000 7f62121e0640 1 -- 172.21.15.7:0/1914753635 wait complete. 2024-09-15T23:36:45.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7f62121e0640 1 Processor -- start 2024-09-15T23:36:45.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de2043640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd4051a10 0x7f8dd40e1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8029c74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 0x7f801c036190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de2043640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4054580 0x7f8dd40d8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de2043640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd4054d80 0x7f8dd4036ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f60cf36d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 0x7f60c8099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de0840640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4054580 0x7f8dd40d8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f577e396640 1 Processor -- start 2024-09-15T23:36:45.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de2043640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd4033330 con 0x7f8dd4054580 2024-09-15T23:36:45.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de0840640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4054580 0x7f8dd40d8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48368/0 (socket says 172.21.15.7:48368) 2024-09-15T23:36:45.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de0840640 1 -- 172.21.15.7:0/845764112 learned_addr learned my addr 172.21.15.7:0/845764112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd400ddc0 con 0x7f8dd4054d80 2024-09-15T23:36:45.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8029c74640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 0x7f801c052550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd40e42f0 con 0x7f8dd4051a10 2024-09-15T23:36:45.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.792+0000 7f62121e0640 1 -- start start 2024-09-15T23:36:45.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8029c74640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c052a90 0x7f801c03ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de1842640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd4054d80 0x7f8dd4036ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8de1041640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd4051a10 0x7f8dd40e1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7fce9ef52640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce980b2570 0x7fce98158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f80237fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 0x7f801c036190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f80237fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 0x7f801c036190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:47312/0 (socket says 172.21.15.7:47312) 2024-09-15T23:36:45.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8023fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c052a90 0x7f801c03ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f8023fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c052a90 0x7f801c03ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48378/0 (socket says 172.21.15.7:48378) 2024-09-15T23:36:45.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f80237fe640 1 -- 172.21.15.7:0/2934147954 learned_addr learned my addr 172.21.15.7:0/2934147954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ef52640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce9815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ef52640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9815b9c0 0x7fce9815d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f801c006bb0 con 0x7f801c052a90 2024-09-15T23:36:45.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f801c0552a0 con 0x7f801c0348b0 2024-09-15T23:36:45.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ef52640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce980b0ce0 con 0x7fce9815b9c0 2024-09-15T23:36:45.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f801c0e3270 con 0x7f801c0e2e90 2024-09-15T23:36:45.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ef52640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fce98073e60 con 0x7fce980b2570 2024-09-15T23:36:45.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ef52640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fce980b0e60 con 0x7fce980b4280 2024-09-15T23:36:45.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f0093227640 1 -- 172.21.15.7:0/466337334 shutdown_connections 2024-09-15T23:36:45.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fbd39065640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd3406edf0 0x7fbd3415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f0093227640 1 --2- 172.21.15.7:0/466337334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c0b40b0 0x7f008c15e3a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f0093227640 1 --2- 172.21.15.7:0/466337334 >> v2:172.21.15.7:3300/0 conn(0x7f008c0b3160 0x7f008c0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f0093227640 1 --2- 172.21.15.7:0/466337334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c154dd0 0x7f008c15b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fbd39065640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd34072600 0x7fbd34099390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f60cf36d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60c80b3b70 0x7f60c8155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f60cf36d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f60c8154dd0 0x7f60c81445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f60cf36d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60c80b1bb0 con 0x7f60c80b2570 2024-09-15T23:36:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f0093227640 1 -- 172.21.15.7:0/466337334 >> 172.21.15.7:0/466337334 conn(0x7f008c072430 msgr2=0x7f008c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f60cd109640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 0x7f60c8099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f60cd109640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 0x7f60c8099900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48388/0 (socket says 172.21.15.7:48388) 2024-09-15T23:36:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f60cd109640 1 -- 172.21.15.7:0/2340679673 learned_addr learned my addr 172.21.15.7:0/2340679673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8073af0 con 0x7f60c80b3b70 2024-09-15T23:36:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ccee640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce980b2570 0x7fce98158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ccee640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce980b2570 0x7fce98158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:47328/0 (socket says 172.21.15.7:47328) 2024-09-15T23:36:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce97fff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce9815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ccee640 1 -- 172.21.15.7:0/1065102986 learned_addr learned my addr 172.21.15.7:0/1065102986 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fbd32d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd3406edf0 0x7fbd3415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f60c80afcd0 con 0x7f60c8154dd0 2024-09-15T23:36:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f0093227640 1 -- 172.21.15.7:0/466337334 shutdown_connections 2024-09-15T23:36:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f0093227640 1 -- 172.21.15.7:0/466337334 wait complete. 2024-09-15T23:36:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fbd32d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd3406edf0 0x7fbd3415f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48410/0 (socket says 172.21.15.7:48410) 2024-09-15T23:36:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8de1842640 1 -- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd4051a10 msgr2=0x7f8dd40e1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8de1842640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd4051a10 0x7f8dd40e1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8de1842640 1 -- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4054580 msgr2=0x7f8dd40d8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8de1842640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4054580 0x7f8dd40d8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8de1842640 1 -- 172.21.15.7:0/845764112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd4039a90 con 0x7f8dd4054d80 2024-09-15T23:36:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fbd32d9d640 1 -- 172.21.15.7:0/1751680075 learned_addr learned my addr 172.21.15.7:0/1751680075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8de0840640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4054580 0x7f8dd40d8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:45.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fbd39065640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbd34165fa0 0x7fbd3409be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8de1041640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd4051a10 0x7f8dd40e1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:45.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fbd3259c640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd34072600 0x7fbd34099390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ccee640 1 -- 172.21.15.7:0/1065102986 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 msgr2=0x7fce9815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3406f200 con 0x7fbd3406edf0 2024-09-15T23:36:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ccee640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce9815cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ccee640 1 -- 172.21.15.7:0/1065102986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9815b9c0 msgr2=0x7fce9815d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ccee640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9815b9c0 0x7fce9815d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.793+0000 7f577e396640 1 -- start start 2024-09-15T23:36:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f577e396640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f577806cc20 0x7f5778162570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f577e396640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57780b5be0 0x7f577816bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7f8de1842640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd4054d80 0x7f8dd4036ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400f570 tx=0x7f8dd80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.794+0000 7fce9ccee640 1 -- 172.21.15.7:0/1065102986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9815e490 con 0x7fce980b2570 2024-09-15T23:36:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f60cd109640 1 -- 172.21.15.7:0/2340679673 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f60c8154dd0 msgr2=0x7f60c81445d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:36:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f60cd109640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f60c8154dd0 0x7f60c81445d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f60cd109640 1 -- 172.21.15.7:0/2340679673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60c80b3b70 msgr2=0x7f60c8155fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:36:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f60cd109640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60c80b3b70 0x7f60c8155fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f60cd109640 1 -- 172.21.15.7:0/2340679673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c81554d0 con 0x7f60c80b2570 2024-09-15T23:36:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f8023fff640 1 -- 172.21.15.7:0/2934147954 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 msgr2=0x7f801c052550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:36:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7fce9ccee640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce980b2570 0x7fce98158160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce8400c400 tx=0x7fce8400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577e396640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5778159dd0 0x7f577816a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577e396640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f577806fd60 con 0x7f57780b5be0 2024-09-15T23:36:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7fbd3359e640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbd34165fa0 0x7fbd3409be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577e396640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57780cf0e0 con 0x7f5778159dd0 2024-09-15T23:36:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577e396640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f57780b6050 con 0x7f577806cc20 2024-09-15T23:36:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbd340b36d0 con 0x7fbd34072600 2024-09-15T23:36:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577cb93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57780b5be0 0x7f577816bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577cb93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57780b5be0 0x7f577816bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48420/0 (socket says 172.21.15.7:48420) 2024-09-15T23:36:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577cb93640 1 -- 172.21.15.7:0/1628637509 learned_addr learned my addr 172.21.15.7:0/1628637509 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f60cd109640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 0x7f60c8099900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60c4000c00 tx=0x7f60c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577d394640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f577806cc20 0x7f5778162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbd340ad140 con 0x7fbd34165fa0 2024-09-15T23:36:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7fce957fa640 1 -- 172.21.15.7:0/1065102986 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84017070 con 0x7fce980b2570 2024-09-15T23:36:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577cb93640 1 -- 172.21.15.7:0/1628637509 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f577806cc20 msgr2=0x7f5778162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f60bdffb640 1 -- 172.21.15.7:0/2340679673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4037050 con 0x7f60c80b2570 2024-09-15T23:36:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577cb93640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f577806cc20 0x7f5778162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577cb93640 1 -- 172.21.15.7:0/1628637509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5778159dd0 msgr2=0x7f577816a360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:36:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577cb93640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5778159dd0 0x7f577816a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f577cb93640 1 -- 172.21.15.7:0/1628637509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577816ab90 con 0x7f57780b5be0 2024-09-15T23:36:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f62121e0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62040538c0 0x7f6204189df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f62121e0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62040e7040 0x7f620401c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f62121e0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62040eab20 0x7f62041852b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7fce9ef52640 1 -- 172.21.15.7:0/1065102986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9815e770 con 0x7fce980b2570 2024-09-15T23:36:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f62121e0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62040070b0 con 0x7f62040e7040 2024-09-15T23:36:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f62121e0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6204007420 con 0x7f62040538c0 2024-09-15T23:36:45.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f62121e0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6204054220 con 0x7f62040eab20 2024-09-15T23:36:45.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c81557b0 con 0x7f60c80b2570 2024-09-15T23:36:45.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f8dd1ffb640 1 -- 172.21.15.7:0/845764112 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8051030 con 0x7f8dd4054d80 2024-09-15T23:36:45.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7fce9ef52640 1 -- 172.21.15.7:0/1065102986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce980ce980 con 0x7fce980b2570 2024-09-15T23:36:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c8099e40 con 0x7f60c80b2570 2024-09-15T23:36:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd4039d70 con 0x7f8dd4054d80 2024-09-15T23:36:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f60bdffb640 1 -- 172.21.15.7:0/2340679673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c4043070 con 0x7f60c80b2570 2024-09-15T23:36:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f60bdffb640 1 -- 172.21.15.7:0/2340679673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40329f0 con 0x7f60c80b2570 2024-09-15T23:36:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f0093227640 1 Processor -- start 2024-09-15T23:36:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f62109dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62040e7040 0x7f620401c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f62109dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62040e7040 0x7f620401c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48430/0 (socket says 172.21.15.7:48430) 2024-09-15T23:36:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f62109dd640 1 -- 172.21.15.7:0/3963885265 learned_addr learned my addr 172.21.15.7:0/3963885265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f0093227640 1 -- start start 2024-09-15T23:36:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7fbd32d9d640 1 -- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbd34165fa0 msgr2=0x7fbd3409be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7fbd32d9d640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbd34165fa0 0x7fbd3409be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7fbd32d9d640 1 -- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd34072600 msgr2=0x7fbd34099390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7fbd32d9d640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd34072600 0x7fbd34099390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f577cb93640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57780b5be0 0x7f577816bc30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f576800b420 tx=0x7f576800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd403a330 con 0x7f8dd4054d80 2024-09-15T23:36:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f8dd1ffb640 1 -- 172.21.15.7:0/845764112 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd804d660 con 0x7f8dd4054d80 2024-09-15T23:36:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f8dd1ffb640 1 -- 172.21.15.7:0/845764112 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80110d0 con 0x7f8dd4054d80 2024-09-15T23:36:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.795+0000 7f8023fff640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 0x7f801c052550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f8023fff640 1 -- 172.21.15.7:0/2934147954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 msgr2=0x7f801c036190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f8023fff640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 0x7f801c036190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f8023fff640 1 -- 172.21.15.7:0/2934147954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c042f80 con 0x7f801c052a90 2024-09-15T23:36:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f8023fff640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c052a90 0x7f801c03ffd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8010031000 tx=0x7f80100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f80237fe640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 0x7f801c036190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f576dffb640 1 -- 172.21.15.7:0/1628637509 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768011830 con 0x7f57780b5be0 2024-09-15T23:36:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f62119df640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62040eab20 0x7f62041852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7f62111de640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62040538c0 0x7f6204189df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f62109dd640 1 -- 172.21.15.7:0/3963885265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62040eab20 msgr2=0x7f62041852b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f62109dd640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62040eab20 0x7f62041852b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f62109dd640 1 -- 172.21.15.7:0/3963885265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62040538c0 msgr2=0x7f6204189df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f62109dd640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62040538c0 0x7f6204189df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f62109dd640 1 -- 172.21.15.7:0/3963885265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620418a330 con 0x7f62040e7040 2024-09-15T23:36:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f800ffff640 1 -- 172.21.15.7:0/2934147954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010041030 con 0x7f801c052a90 2024-09-15T23:36:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f800ffff640 1 -- 172.21.15.7:0/2934147954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f801003d420 con 0x7f801c052a90 2024-09-15T23:36:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f800ffff640 1 -- 172.21.15.7:0/2934147954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010037040 con 0x7f801c052a90 2024-09-15T23:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7fce957fa640 1 -- 172.21.15.7:0/1065102986 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce840052d0 con 0x7fce980b2570 2024-09-15T23:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7fce957fa640 1 -- 172.21.15.7:0/1065102986 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84003830 con 0x7fce980b2570 2024-09-15T23:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.796+0000 7fbd32d9d640 1 -- 172.21.15.7:0/1751680075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd34201e50 con 0x7fbd3406edf0 2024-09-15T23:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f0093227640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c0b3160 0x7f008c09acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f0093227640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c0b40b0 0x7f008c159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f0093227640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008c154dd0 0x7f008c159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f00917c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008c154dd0 0x7f008c159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f00917c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008c154dd0 0x7f008c159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48440/0 (socket says 172.21.15.7:48440) 2024-09-15T23:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f00917c4640 1 -- 172.21.15.7:0/3764619381 learned_addr learned my addr 172.21.15.7:0/3764619381 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f008c0b35c0 con 0x7f008c154dd0 2024-09-15T23:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1551b0 con 0x7f008c0b3160 2024-09-15T23:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f008c0b2570 con 0x7f008c0b40b0 2024-09-15T23:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7fce9ef52640 1 -- 172.21.15.7:0/1065102986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce98154dd0 con 0x7fce980b2570 2024-09-15T23:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c0368c0 con 0x7f801c052a90 2024-09-15T23:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c036e80 con 0x7f801c052a90 2024-09-15T23:36:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7fbd32d9d640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd3406edf0 0x7fbd3415f6b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c450 tx=0x7fbd2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7fbd3259c640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd34072600 0x7fbd34099390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f0090fc3640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c0b3160 0x7f008c09acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.798+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dbc005290 con 0x7f8dd4054d80 2024-09-15T23:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577816ae70 con 0x7f57780b5be0 2024-09-15T23:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.797+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577816b430 con 0x7f57780b5be0 2024-09-15T23:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f576dffb640 1 -- 172.21.15.7:0/1628637509 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5768016070 con 0x7f57780b5be0 2024-09-15T23:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f576dffb640 1 -- 172.21.15.7:0/1628637509 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768011c10 con 0x7f57780b5be0 2024-09-15T23:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f62109dd640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62040e7040 0x7f620401c970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f620c06b070 tx=0x7f620c069750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7fce957fa640 1 -- 172.21.15.7:0/1065102986 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fce8401a070 con 0x7fce980b2570 2024-09-15T23:36:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f008bfff640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c0b40b0 0x7f008c159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f00917c4640 1 -- 172.21.15.7:0/3764619381 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c0b40b0 msgr2=0x7f008c159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f00917c4640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c0b40b0 0x7f008c159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f00917c4640 1 -- 172.21.15.7:0/3764619381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c0b3160 msgr2=0x7f008c09acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f00917c4640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c0b3160 0x7f008c09acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f00917c4640 1 -- 172.21.15.7:0/3764619381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c15efd0 con 0x7f008c154dd0 2024-09-15T23:36:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f8dd1ffb640 1 -- 172.21.15.7:0/845764112 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8dd8054070 con 0x7f8dd4054d80 2024-09-15T23:36:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f008bfff640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c0b40b0 0x7f008c159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a8005290 con 0x7f60c80b2570 2024-09-15T23:36:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.799+0000 7f00917c4640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008c154dd0 0x7f008c159f40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00840079e0 tx=0x7f0084004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7fce957fa640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fce7c06c330 0x7fce7c06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c035f80 con 0x7f801c052a90 2024-09-15T23:36:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5758005290 con 0x7f57780b5be0 2024-09-15T23:36:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7fce957fa640 1 -- 172.21.15.7:0/1065102986 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fce8408aa40 con 0x7fce980b2570 2024-09-15T23:36:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7f8dd1ffb640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dc006c380 0x7f8dc006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7fce97fff640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fce7c06c330 0x7fce7c06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7f8dd1ffb640 1 -- 172.21.15.7:0/845764112 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8dd80bcc30 con 0x7f8dd4054d80 2024-09-15T23:36:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7f8de1041640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dc006c380 0x7f8dc006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7fce97fff640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fce7c06c330 0x7fce7c06e7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce98154ca0 tx=0x7fce8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7fbd1b7fe640 1 -- 172.21.15.7:0/1751680075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240052d0 con 0x7fbd3406edf0 2024-09-15T23:36:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7fbd1b7fe640 1 -- 172.21.15.7:0/1751680075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd24005470 con 0x7fbd3406edf0 2024-09-15T23:36:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7fbd1b7fe640 1 -- 172.21.15.7:0/1751680075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24012850 con 0x7fbd3406edf0 2024-09-15T23:36:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.800+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd34202130 con 0x7fbd3406edf0 2024-09-15T23:36:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.801+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd342026f0 con 0x7fbd3406edf0 2024-09-15T23:36:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.801+0000 7f8de1041640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dc006c380 0x7f8dc006e840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc06af90 tx=0x7f8ddc06f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.801+0000 7f00897fa640 1 -- 172.21.15.7:0/3764619381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084041020 con 0x7f008c154dd0 2024-09-15T23:36:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.801+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008c1ef9b0 con 0x7f008c154dd0 2024-09-15T23:36:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.801+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f008c1eff70 con 0x7f008c154dd0 2024-09-15T23:36:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.801+0000 7f6201ffb640 1 -- 172.21.15.7:0/3963885265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c094690 con 0x7f62040e7040 2024-09-15T23:36:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.801+0000 7f800ffff640 1 -- 172.21.15.7:0/2934147954 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8010044070 con 0x7f801c052a90 2024-09-15T23:36:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.802+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62040e4850 con 0x7f62040e7040 2024-09-15T23:36:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.802+0000 7f60bdffb640 1 -- 172.21.15.7:0/2340679673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f60c4032b90 con 0x7f60c80b2570 2024-09-15T23:36:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.802+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620404bd10 con 0x7f62040e7040 2024-09-15T23:36:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.802+0000 7f800ffff640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f800806c3b0 0x7f800806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.802+0000 7f800ffff640 1 -- 172.21.15.7:0/2934147954 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f80100ac430 con 0x7f801c052a90 2024-09-15T23:36:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.802+0000 7f60bdffb640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60ac06c2b0 0x7f60ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.802+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3406e8b0 con 0x7fbd3406edf0 2024-09-15T23:36:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7f60bdffb640 1 -- 172.21.15.7:0/2340679673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f60c40ad110 con 0x7f60c80b2570 2024-09-15T23:36:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7f576dffb640 1 -- 172.21.15.7:0/1628637509 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5768004020 con 0x7f57780b5be0 2024-09-15T23:36:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7f60cc908640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60ac06c2b0 0x7f60ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7f80237fe640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f800806c3b0 0x7f800806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008c0b3b70 con 0x7f008c154dd0 2024-09-15T23:36:45.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7f576dffb640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f575c06c300 0x7f575c06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7f60cc908640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60ac06c2b0 0x7f60ac06e770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f60c8146ca0 tx=0x7f60c0009230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7f80237fe640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f800806c3b0 0x7f800806e870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f801c046630 tx=0x7f8018009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7fce957fa640 1 -- 172.21.15.7:0/1065102986 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fce84058900 con 0x7fce980b2570 2024-09-15T23:36:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.803+0000 7f577d394640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f575c06c300 0x7f575c06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.804+0000 7f577d394640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f575c06c300 0x7f575c06e7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5774034000 tx=0x7f577402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.804+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6204047690 con 0x7f62040e7040 2024-09-15T23:36:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.807+0000 7f800ffff640 1 -- 172.21.15.7:0/2934147954 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f801007a840 con 0x7f801c052a90 2024-09-15T23:36:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.808+0000 7f8dd1ffb640 1 -- 172.21.15.7:0/845764112 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8dd808b040 con 0x7f8dd4054d80 2024-09-15T23:36:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.812+0000 7f60bdffb640 1 -- 172.21.15.7:0/2340679673 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f60c80b44c0 con 0x7f60c80b2570 2024-09-15T23:36:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.812+0000 7f00897fa640 1 -- 172.21.15.7:0/3764619381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0084037040 con 0x7f008c154dd0 2024-09-15T23:36:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.812+0000 7f00897fa640 1 -- 172.21.15.7:0/3764619381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084036070 con 0x7f008c154dd0 2024-09-15T23:36:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.813+0000 7f576dffb640 1 -- 172.21.15.7:0/1628637509 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5768010e60 con 0x7f57780b5be0 2024-09-15T23:36:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.813+0000 7fbd1b7fe640 1 -- 172.21.15.7:0/1751680075 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbd240129f0 con 0x7fbd3406edf0 2024-09-15T23:36:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.813+0000 7f6201ffb640 1 -- 172.21.15.7:0/3963885265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c08f070 con 0x7f62040e7040 2024-09-15T23:36:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.813+0000 7f6201ffb640 1 -- 172.21.15.7:0/3963885265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c094a70 con 0x7f62040e7040 2024-09-15T23:36:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.814+0000 7fbd1b7fe640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbd2006c2e0 0x7fbd2006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.814+0000 7fbd1b7fe640 1 -- 172.21.15.7:0/1751680075 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbd2408bcb0 con 0x7fbd3406edf0 2024-09-15T23:36:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.814+0000 7fbd3259c640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbd2006c2e0 0x7fbd2006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.815+0000 7fbd3259c640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbd2006c2e0 0x7fbd2006e7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c033000 tx=0x7fbd2c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.819+0000 7f576dffb640 1 -- 172.21.15.7:0/1628637509 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f576805a2e0 con 0x7f57780b5be0 2024-09-15T23:36:45.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.821+0000 7f6201ffb640 1 -- 172.21.15.7:0/3963885265 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f620c09d3f0 con 0x7f62040e7040 2024-09-15T23:36:45.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.821+0000 7f6201ffb640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f61f006c2b0 0x7f61f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.822+0000 7f62111de640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f61f006c2b0 0x7f61f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.822+0000 7f00897fa640 1 -- 172.21.15.7:0/3764619381 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0084044020 con 0x7f008c154dd0 2024-09-15T23:36:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.822+0000 7f00897fa640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006c06c2b0 0x7f006c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.823+0000 7f62111de640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f61f006c2b0 0x7f61f006e770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6208033000 tx=0x7f620802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.823+0000 7f0090fc3640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006c06c2b0 0x7f006c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.823+0000 7f0090fc3640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006c06c2b0 0x7f006c06e770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f007400a950 tx=0x7f0074008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.828+0000 7f00897fa640 1 -- 172.21.15.7:0/3764619381 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0084009070 con 0x7f008c154dd0 2024-09-15T23:36:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.828+0000 7f6201ffb640 1 -- 172.21.15.7:0/3963885265 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f620c0d3590 con 0x7f62040e7040 2024-09-15T23:36:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.828+0000 7fbd1b7fe640 1 -- 172.21.15.7:0/1751680075 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbd24059b00 con 0x7fbd3406edf0 2024-09-15T23:36:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.837+0000 7f6201ffb640 1 -- 172.21.15.7:0/3963885265 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6204047690 con 0x7f62040e7040 2024-09-15T23:36:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:45.846+0000 7f00897fa640 1 -- 172.21.15.7:0/3764619381 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f008404b070 con 0x7f008c154dd0 2024-09-15T23:36:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.204+0000 7fce9ef52640 1 -- 172.21.15.7:0/1065102986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fce980ceb50 con 0x7fce980b2570 2024-09-15T23:36:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.204+0000 7fce957fa640 1 -- 172.21.15.7:0/1065102986 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce84088e20 con 0x7fce980b2570 2024-09-15T23:36:46.207 INFO:teuthology.orchestra.run.smithi007.stdout:60129542152 2024-09-15T23:36:46.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.208+0000 7fce82ffd640 1 -- 172.21.15.7:0/1065102986 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fce7c06c330 msgr2=0x7fce7c06e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.208+0000 7fce82ffd640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fce7c06c330 0x7fce7c06e7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce98154ca0 tx=0x7fce8c009040 comp rx=0 tx=0).stop 2024-09-15T23:36:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.209+0000 7fce82ffd640 1 -- 172.21.15.7:0/1065102986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce980b2570 msgr2=0x7fce98158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.209+0000 7fce82ffd640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce980b2570 0x7fce98158160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce8400c400 tx=0x7fce8400c8c0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.209+0000 7fce82ffd640 1 -- 172.21.15.7:0/1065102986 shutdown_connections 2024-09-15T23:36:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.209+0000 7fce82ffd640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9815b9c0 0x7fce9815d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.209+0000 7fce82ffd640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fce7c06c330 0x7fce7c06e7f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.209+0000 7fce82ffd640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fce980b4280 0x7fce9815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.209+0000 7fce82ffd640 1 --2- 172.21.15.7:0/1065102986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce980b2570 0x7fce98158160 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.209+0000 7fce82ffd640 1 -- 172.21.15.7:0/1065102986 >> 172.21.15.7:0/1065102986 conn(0x7fce98072430 msgr2=0x7fce98159070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.210+0000 7fce82ffd640 1 -- 172.21.15.7:0/1065102986 shutdown_connections 2024-09-15T23:36:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.210+0000 7fce82ffd640 1 -- 172.21.15.7:0/1065102986 wait complete. 2024-09-15T23:36:46.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.220+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f60a8005850 con 0x7f60c80b2570 2024-09-15T23:36:46.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.220+0000 7f60bdffb640 1 -- 172.21.15.7:0/2340679673 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f60c407b530 con 0x7f60c80b2570 2024-09-15T23:36:46.222 INFO:teuthology.orchestra.run.smithi007.stdout:55834574858 2024-09-15T23:36:46.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.221+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f801c007120 con 0x7f801c052a90 2024-09-15T23:36:46.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.221+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8dbc005580 con 0x7f8dd4054d80 2024-09-15T23:36:46.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.221+0000 7f800ffff640 1 -- 172.21.15.7:0/2934147954 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f801007a140 con 0x7f801c052a90 2024-09-15T23:36:46.223 INFO:teuthology.orchestra.run.smithi007.stdout:64424509449 2024-09-15T23:36:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.222+0000 7f8dd1ffb640 1 -- 172.21.15.7:0/845764112 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8dd808a940 con 0x7f8dd4054d80 2024-09-15T23:36:46.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.224+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60ac06c2b0 msgr2=0x7f60ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.224+0000 7f60cf36d640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60ac06c2b0 0x7f60ac06e770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f60c8146ca0 tx=0x7f60c0009230 comp rx=0 tx=0).stop 2024-09-15T23:36:46.226 INFO:teuthology.orchestra.run.smithi007.stdout:64424509450 2024-09-15T23:36:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.224+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 msgr2=0x7f60c8099900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.224+0000 7f60cf36d640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 0x7f60c8099900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60c4000c00 tx=0x7f60c40046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.224+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 shutdown_connections 2024-09-15T23:36:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.224+0000 7f60cf36d640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60ac06c2b0 0x7f60ac06e770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.224+0000 7f60cf36d640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f60c8154dd0 0x7f60c81445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.224+0000 7f60cf36d640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60c80b3b70 0x7f60c8155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.224+0000 7f60cf36d640 1 --2- 172.21.15.7:0/2340679673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c80b2570 0x7f60c8099900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 >> 172.21.15.7:0/2340679673 conn(0x7f60c8072430 msgr2=0x7f60c8159a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f800806c3b0 msgr2=0x7f800806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 shutdown_connections 2024-09-15T23:36:46.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f800806c3b0 0x7f800806e870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f801c046630 tx=0x7f8018009110 comp rx=0 tx=0).stop 2024-09-15T23:36:46.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c052a90 msgr2=0x7f801c03ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c052a90 0x7f801c03ffd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8010031000 tx=0x7f80100046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f60cf36d640 1 -- 172.21.15.7:0/2340679673 wait complete. 2024-09-15T23:36:46.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 shutdown_connections 2024-09-15T23:36:46.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c052a90 0x7f801c03ffd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f800806c3b0 0x7f800806e870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f801c0e2e90 0x7f801c052550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 --2- 172.21.15.7:0/2934147954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f801c0348b0 0x7f801c036190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 >> 172.21.15.7:0/2934147954 conn(0x7f801c01ea50 msgr2=0x7f801c0335f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dc006c380 msgr2=0x7f8dc006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8de2043640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dc006c380 0x7f8dc006e840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc06af90 tx=0x7f8ddc06f040 comp rx=0 tx=0).stop 2024-09-15T23:36:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd4054d80 msgr2=0x7f8dd4036ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.225+0000 7f8de2043640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd4054d80 0x7f8dd4036ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400f570 tx=0x7f8dd80046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 shutdown_connections 2024-09-15T23:36:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8de2043640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd4054d80 0x7f8dd4036ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8de2043640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd4054580 0x7f8dd40d8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8de2043640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dc006c380 0x7f8dc006e840 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc06af90 tx=0x7f8ddc06f040 comp rx=0 tx=0).stop 2024-09-15T23:36:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8de2043640 1 --2- 172.21.15.7:0/845764112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd4051a10 0x7f8dd40e1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 >> 172.21.15.7:0/845764112 conn(0x7f8dd40567a0 msgr2=0x7f8dd40df9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 shutdown_connections 2024-09-15T23:36:46.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8029c74640 1 -- 172.21.15.7:0/2934147954 wait complete. 2024-09-15T23:36:46.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 shutdown_connections 2024-09-15T23:36:46.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7f8de2043640 1 -- 172.21.15.7:0/845764112 wait complete. 2024-09-15T23:36:46.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.226+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fbd340d0d50 con 0x7fbd3406edf0 2024-09-15T23:36:46.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.7 2024-09-15T23:36:46.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.227+0000 7fbd1b7fe640 1 -- 172.21.15.7:0/1751680075 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd2405de00 con 0x7fbd3406edf0 2024-09-15T23:36:46.233 INFO:teuthology.orchestra.run.smithi007.stdout:64424509449 2024-09-15T23:36:46.234 DEBUG:teuthology.parallel:result is None 2024-09-15T23:36:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.229+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbd2006c2e0 msgr2=0x7fbd2006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.229+0000 7fbd39065640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbd2006c2e0 0x7fbd2006e7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c033000 tx=0x7fbd2c009000 comp rx=0 tx=0).stop 2024-09-15T23:36:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.229+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd3406edf0 msgr2=0x7fbd3415f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.229+0000 7fbd39065640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd3406edf0 0x7fbd3415f6b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c450 tx=0x7fbd2400c910 comp rx=0 tx=0).stop 2024-09-15T23:36:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.230+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 shutdown_connections 2024-09-15T23:36:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.230+0000 7fbd39065640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbd2006c2e0 0x7fbd2006e7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.230+0000 7fbd39065640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbd34165fa0 0x7fbd3409be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.230+0000 7fbd39065640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd34072600 0x7fbd34099390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.230+0000 7fbd39065640 1 --2- 172.21.15.7:0/1751680075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd3406edf0 0x7fbd3415f6b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.230+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 >> 172.21.15.7:0/1751680075 conn(0x7fbd340b1e60 msgr2=0x7fbd340ad4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.230+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 shutdown_connections 2024-09-15T23:36:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.230+0000 7fbd39065640 1 -- 172.21.15.7:0/1751680075 wait complete. 2024-09-15T23:36:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.243+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5758005850 con 0x7f57780b5be0 2024-09-15T23:36:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.244+0000 7f576dffb640 1 -- 172.21.15.7:0/1628637509 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5768059be0 con 0x7f57780b5be0 2024-09-15T23:36:46.246 INFO:teuthology.orchestra.run.smithi007.stdout:64424509449 2024-09-15T23:36:46.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.1 2024-09-15T23:36:46.247 DEBUG:teuthology.parallel:result is None 2024-09-15T23:36:46.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-09-15T23:36:46.248 DEBUG:teuthology.parallel:result is None 2024-09-15T23:36:46.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-09-15T23:36:46.249 DEBUG:teuthology.parallel:result is None 2024-09-15T23:36:46.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.2 2024-09-15T23:36:46.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.246+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6204047970 con 0x7f62040e7040 2024-09-15T23:36:46.250 DEBUG:teuthology.parallel:result is None 2024-09-15T23:36:46.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.246+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f575c06c300 msgr2=0x7f575c06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.246+0000 7f577e396640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f575c06c300 0x7f575c06e7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5774034000 tx=0x7f577402e000 comp rx=0 tx=0).stop 2024-09-15T23:36:46.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.246+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57780b5be0 msgr2=0x7f577816bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.246+0000 7f577e396640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57780b5be0 0x7f577816bc30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f576800b420 tx=0x7f576800b8e0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.246+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 shutdown_connections 2024-09-15T23:36:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.246+0000 7f577e396640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5778159dd0 0x7f577816a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.247+0000 7f577e396640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57780b5be0 0x7f577816bc30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.247+0000 7f577e396640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f575c06c300 0x7f575c06e7c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.247+0000 7f577e396640 1 --2- 172.21.15.7:0/1628637509 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f577806cc20 0x7f5778162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.247+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 >> 172.21.15.7:0/1628637509 conn(0x7f5778072640 msgr2=0x7f57780cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.247+0000 7f6201ffb640 1 -- 172.21.15.7:0/3963885265 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6204047970 con 0x7f62040e7040 2024-09-15T23:36:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.247+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 shutdown_connections 2024-09-15T23:36:46.252 INFO:teuthology.orchestra.run.smithi007.stdout:55834574857 2024-09-15T23:36:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.247+0000 7f577e396640 1 -- 172.21.15.7:0/1628637509 wait complete. 2024-09-15T23:36:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.247+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f008c06ced0 con 0x7f008c154dd0 2024-09-15T23:36:46.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.248+0000 7f00897fa640 1 -- 172.21.15.7:0/3764619381 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f008402ae20 con 0x7f008c154dd0 2024-09-15T23:36:46.253 INFO:teuthology.orchestra.run.smithi007.stdout:60129542152 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.249+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f61f006c2b0 msgr2=0x7f61f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f61f006c2b0 0x7f61f006e770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6208033000 tx=0x7f620802e000 comp rx=0 tx=0).stop 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62040e7040 msgr2=0x7f620401c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62040e7040 0x7f620401c970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f620c06b070 tx=0x7f620c069750 comp rx=0 tx=0).stop 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 shutdown_connections 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f61f006c2b0 0x7f61f006e770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62040eab20 0x7f62041852b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62040e7040 0x7f620401c970 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 --2- 172.21.15.7:0/3963885265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62040538c0 0x7f6204189df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 >> 172.21.15.7:0/3963885265 conn(0x7f6204036090 msgr2=0x7f6204034000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 shutdown_connections 2024-09-15T23:36:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f62121e0640 1 -- 172.21.15.7:0/3963885265 wait complete. 2024-09-15T23:36:46.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006c06c2b0 msgr2=0x7f006c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006c06c2b0 0x7f006c06e770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f007400a950 tx=0x7f0074008040 comp rx=0 tx=0).stop 2024-09-15T23:36:46.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008c154dd0 msgr2=0x7f008c159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008c154dd0 0x7f008c159f40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00840079e0 tx=0x7f0084004a40 comp rx=0 tx=0).stop 2024-09-15T23:36:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 shutdown_connections 2024-09-15T23:36:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008c154dd0 0x7f008c159f40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006c06c2b0 0x7f006c06e770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f008c0b40b0 0x7f008c159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 --2- 172.21.15.7:0/3764619381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c0b3160 0x7f008c09acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 >> 172.21.15.7:0/3764619381 conn(0x7f008c072430 msgr2=0x7f008c15bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 shutdown_connections 2024-09-15T23:36:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.250+0000 7f0093227640 1 -- 172.21.15.7:0/3764619381 wait complete. 2024-09-15T23:36:46.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.0 2024-09-15T23:36:46.262 DEBUG:teuthology.parallel:result is None 2024-09-15T23:36:46.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.6 2024-09-15T23:36:46.265 DEBUG:teuthology.parallel:result is None 2024-09-15T23:36:46.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-09-15T23:36:46.265 DEBUG:teuthology.parallel:result is None 2024-09-15T23:36:46.266 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T23:36:46.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:36:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.405+0000 7f355a742640 1 Processor -- start 2024-09-15T23:36:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.405+0000 7f355a742640 1 -- start start 2024-09-15T23:36:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.406+0000 7f355a742640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35540d0f90 0x7f35540d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.406+0000 7f355a742640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35540d0a30 con 0x7f35540d52e0 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.406+0000 7f355a742640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f355406dd80 con 0x7f355415e840 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.406+0000 7f355a742640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f35540cfe90 con 0x7f35540d0f90 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.406+0000 7f3558cdf640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35540d0f90 0x7f35540d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.406+0000 7f35537fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f35540d52e0 0x7f355406d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44734/0 (socket says 172.21.15.7:44734) 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.406+0000 7f35537fe640 1 -- 172.21.15.7:0/2151675627 learned_addr learned my addr 172.21.15.7:0/2151675627 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.406+0000 7f3558cdf640 1 -- 172.21.15.7:0/2151675627 >> v1:172.21.15.18:6789/0 conn(0x7f355415e840 legacy=0x7f35540d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.406+0000 7f3558cdf640 1 -- 172.21.15.7:0/2151675627 >> v1:172.21.15.7:6789/0 conn(0x7f35540d52e0 legacy=0x7f355406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.407+0000 7f3552ffd640 1 -- 172.21.15.7:0/2151675627 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 436750596 0 0) 0x7f35540d0a30 con 0x7f35540d52e0 2024-09-15T23:36:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.407+0000 7f3558cdf640 1 -- 172.21.15.7:0/2151675627 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35540cf560 con 0x7f35540d0f90 2024-09-15T23:36:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.407+0000 7f3558cdf640 1 --2- 172.21.15.7:0/2151675627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 0x7f35540d13c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f35540d0010 tx=0x7f354c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2281ff1a7f1ef3db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.407+0000 7f3552ffd640 1 -- 172.21.15.7:0/2151675627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c007740 con 0x7f35540d0f90 2024-09-15T23:36:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.407+0000 7f3552ffd640 1 -- 172.21.15.7:0/2151675627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354c0078e0 con 0x7f35540d0f90 2024-09-15T23:36:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.407+0000 7f3552ffd640 1 -- 172.21.15.7:0/2151675627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c007cb0 con 0x7f35540d0f90 2024-09-15T23:36:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.408+0000 7f355a742640 1 -- 172.21.15.7:0/2151675627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 msgr2=0x7f35540d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.408+0000 7f355a742640 1 --2- 172.21.15.7:0/2151675627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 0x7f35540d13c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f35540d0010 tx=0x7f354c02a6f0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.408+0000 7f355a742640 1 -- 172.21.15.7:0/2151675627 shutdown_connections 2024-09-15T23:36:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.408+0000 7f355a742640 1 --2- 172.21.15.7:0/2151675627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 0x7f35540d13c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.408+0000 7f355a742640 1 -- 172.21.15.7:0/2151675627 >> 172.21.15.7:0/2151675627 conn(0x7f355408bfd0 msgr2=0x7f35540d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.408+0000 7f355a742640 1 -- 172.21.15.7:0/2151675627 shutdown_connections 2024-09-15T23:36:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.408+0000 7f355a742640 1 -- 172.21.15.7:0/2151675627 wait complete. 2024-09-15T23:36:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.409+0000 7f355a742640 1 Processor -- start 2024-09-15T23:36:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.409+0000 7f355a742640 1 -- start start 2024-09-15T23:36:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f355a742640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 0x7f35540b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f355a742640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f35540d52e0 0x7f35541ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f355a742640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f355415e840 0x7f35541f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f355a742640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35540cfb20 con 0x7f35540d0f90 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f355a742640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f35540d1900 con 0x7f355415e840 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f355a742640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f355406e030 con 0x7f35540d52e0 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f3553fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 0x7f35540b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f3553fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 0x7f35540b6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48462/0 (socket says 172.21.15.7:48462) 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f3553fff640 1 -- 172.21.15.7:0/186566069 learned_addr learned my addr 172.21.15.7:0/186566069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f35537fe640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f35540d52e0 0x7f35541ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.410+0000 7f3558cdf640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f355415e840 0x7f35541f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f3553fff640 1 -- 172.21.15.7:0/186566069 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f35540d52e0 msgr2=0x7f35541ee2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f3553fff640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f35540d52e0 0x7f35541ee2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f3553fff640 1 -- 172.21.15.7:0/186566069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f355415e840 msgr2=0x7f35541f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f3553fff640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f355415e840 0x7f35541f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f3553fff640 1 -- 172.21.15.7:0/186566069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35540b4c80 con 0x7f35540d0f90 2024-09-15T23:36:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f3553fff640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 0x7f35540b6870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f354400c400 tx=0x7f354400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f3550ff9640 1 -- 172.21.15.7:0/186566069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544017070 con 0x7f35540d0f90 2024-09-15T23:36:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f3550ff9640 1 -- 172.21.15.7:0/186566069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35440052d0 con 0x7f35540d0f90 2024-09-15T23:36:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35540b4f60 con 0x7f35540d0f90 2024-09-15T23:36:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f3550ff9640 1 -- 172.21.15.7:0/186566069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544003830 con 0x7f35540d0f90 2024-09-15T23:36:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.411+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35540b5520 con 0x7f35540d0f90 2024-09-15T23:36:46.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.412+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355415ec20 con 0x7f35540d0f90 2024-09-15T23:36:46.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.412+0000 7f3550ff9640 1 -- 172.21.15.7:0/186566069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f354401a070 con 0x7f35540d0f90 2024-09-15T23:36:46.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.413+0000 7f3550ff9640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f354006c2e0 0x7f354006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.413+0000 7f3550ff9640 1 -- 172.21.15.7:0/186566069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f354408aa10 con 0x7f35540d0f90 2024-09-15T23:36:46.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.413+0000 7f35537fe640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f354006c2e0 0x7f354006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.413+0000 7f35537fe640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f354006c2e0 0x7f354006e7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3548007fd0 tx=0x7f3548009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:46.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.417+0000 7f3550ff9640 1 -- 172.21.15.7:0/186566069 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35440587f0 con 0x7f35540d0f90 2024-09-15T23:36:46.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.590+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35540d1cb0 con 0x7f354006c2e0 2024-09-15T23:36:46.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.598+0000 7f3550ff9640 1 -- 172.21.15.7:0/186566069 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7f35540d1cb0 con 0x7f354006c2e0 2024-09-15T23:36:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:46.605 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f354006c2e0 msgr2=0x7f354006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f354006c2e0 0x7f354006e7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3548007fd0 tx=0x7f3548009040 comp rx=0 tx=0).stop 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 msgr2=0x7f35540b6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 0x7f35540b6870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f354400c400 tx=0x7f354400c8c0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 shutdown_connections 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f355415e840 0x7f35541f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f354006c2e0 0x7f354006e7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f35540d52e0 0x7f35541ee2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 --2- 172.21.15.7:0/186566069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35540d0f90 0x7f35540b6870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 >> 172.21.15.7:0/186566069 conn(0x7f355408bfd0 msgr2=0x7f35540b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 shutdown_connections 2024-09-15T23:36:46.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.601+0000 7f355a742640 1 -- 172.21.15.7:0/186566069 wait complete. 2024-09-15T23:36:46.614 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-15T23:36:45.816409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252752,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721968,"statfs":{"total":773094113280,"available":772835295232,"internally_reserved":0,"allocated":3317760,"data_stored":2012508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003016"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623447+0000","last_change":"2024-09-15T23:36:22.558162+0000","last_active":"2024-09-15T23:36:28.623447+0000","last_peered":"2024-09-15T23:36:28.623447+0000","last_clean":"2024-09-15T23:36:28.623447+0000","last_became_active":"2024-09-15T23:36:22.557752+0000","last_became_peered":"2024-09-15T23:36:22.557752+0000","last_unstale":"2024-09-15T23:36:28.623447+0000","last_undegraded":"2024-09-15T23:36:28.623447+0000","last_fullsized":"2024-09-15T23:36:28.623447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:17.806236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024436+0000","last_change":"2024-09-15T23:36:23.591313+0000","last_active":"2024-09-15T23:36:30.024436+0000","last_peered":"2024-09-15T23:36:30.024436+0000","last_clean":"2024-09-15T23:36:30.024436+0000","last_became_active":"2024-09-15T23:36:23.590422+0000","last_became_peered":"2024-09-15T23:36:23.590422+0000","last_unstale":"2024-09-15T23:36:30.024436+0000","last_undegraded":"2024-09-15T23:36:30.024436+0000","last_fullsized":"2024-09-15T23:36:30.024436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454338+0000","last_change":"2024-09-15T23:36:27.609388+0000","last_active":"2024-09-15T23:36:29.454338+0000","last_peered":"2024-09-15T23:36:29.454338+0000","last_clean":"2024-09-15T23:36:29.454338+0000","last_became_active":"2024-09-15T23:36:27.608999+0000","last_became_peered":"2024-09-15T23:36:27.608999+0000","last_unstale":"2024-09-15T23:36:29.454338+0000","last_undegraded":"2024-09-15T23:36:29.454338+0000","last_fullsized":"2024-09-15T23:36:29.454338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:30.088730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024564+0000","last_change":"2024-09-15T23:36:22.562097+0000","last_active":"2024-09-15T23:36:30.024564+0000","last_peered":"2024-09-15T23:36:30.024564+0000","last_clean":"2024-09-15T23:36:30.024564+0000","last_became_active":"2024-09-15T23:36:22.561356+0000","last_became_peered":"2024-09-15T23:36:22.561356+0000","last_unstale":"2024-09-15T23:36:30.024564+0000","last_undegraded":"2024-09-15T23:36:30.024564+0000","last_fullsized":"2024-09-15T23:36:30.024564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:46.482072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024461+0000","last_change":"2024-09-15T23:36:23.589236+0000","last_active":"2024-09-15T23:36:30.024461+0000","last_peered":"2024-09-15T23:36:30.024461+0000","last_clean":"2024-09-15T23:36:30.024461+0000","last_became_active":"2024-09-15T23:36:23.588873+0000","last_became_peered":"2024-09-15T23:36:23.588873+0000","last_unstale":"2024-09-15T23:36:30.024461+0000","last_undegraded":"2024-09-15T23:36:30.024461+0000","last_fullsized":"2024-09-15T23:36:30.024461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:18.209937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024371+0000","last_change":"2024-09-15T23:36:27.611526+0000","last_active":"2024-09-15T23:36:30.024371+0000","last_peered":"2024-09-15T23:36:30.024371+0000","last_clean":"2024-09-15T23:36:30.024371+0000","last_became_active":"2024-09-15T23:36:27.610759+0000","last_became_peered":"2024-09-15T23:36:27.610759+0000","last_unstale":"2024-09-15T23:36:30.024371+0000","last_undegraded":"2024-09-15T23:36:30.024371+0000","last_fullsized":"2024-09-15T23:36:30.024371+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:06.783968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025217+0000","last_change":"2024-09-15T23:36:22.568198+0000","last_active":"2024-09-15T23:36:30.025217+0000","last_peered":"2024-09-15T23:36:30.025217+0000","last_clean":"2024-09-15T23:36:30.025217+0000","last_became_active":"2024-09-15T23:36:22.567888+0000","last_became_peered":"2024-09-15T23:36:22.567888+0000","last_unstale":"2024-09-15T23:36:30.025217+0000","last_undegraded":"2024-09-15T23:36:30.025217+0000","last_fullsized":"2024-09-15T23:36:30.025217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:14:47.610942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024413+0000","last_change":"2024-09-15T23:36:23.593279+0000","last_active":"2024-09-15T23:36:30.024413+0000","last_peered":"2024-09-15T23:36:30.024413+0000","last_clean":"2024-09-15T23:36:30.024413+0000","last_became_active":"2024-09-15T23:36:23.592921+0000","last_became_peered":"2024-09-15T23:36:23.592921+0000","last_unstale":"2024-09-15T23:36:30.024413+0000","last_undegraded":"2024-09-15T23:36:30.024413+0000","last_fullsized":"2024-09-15T23:36:30.024413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:40.194400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024320+0000","last_change":"2024-09-15T23:36:27.612903+0000","last_active":"2024-09-15T23:36:30.024320+0000","last_peered":"2024-09-15T23:36:30.024320+0000","last_clean":"2024-09-15T23:36:30.024320+0000","last_became_active":"2024-09-15T23:36:27.612441+0000","last_became_peered":"2024-09-15T23:36:27.612441+0000","last_unstale":"2024-09-15T23:36:30.024320+0000","last_undegraded":"2024-09-15T23:36:30.024320+0000","last_fullsized":"2024-09-15T23:36:30.024320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:43.647541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025323+0000","last_change":"2024-09-15T23:36:22.566488+0000","last_active":"2024-09-15T23:36:30.025323+0000","last_peered":"2024-09-15T23:36:30.025323+0000","last_clean":"2024-09-15T23:36:30.025323+0000","last_became_active":"2024-09-15T23:36:22.566016+0000","last_became_peered":"2024-09-15T23:36:22.566016+0000","last_unstale":"2024-09-15T23:36:30.025323+0000","last_undegraded":"2024-09-15T23:36:30.025323+0000","last_fullsized":"2024-09-15T23:36:30.025323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:45.276407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622706+0000","last_change":"2024-09-15T23:36:23.590254+0000","last_active":"2024-09-15T23:36:28.622706+0000","last_peered":"2024-09-15T23:36:28.622706+0000","last_clean":"2024-09-15T23:36:28.622706+0000","last_became_active":"2024-09-15T23:36:23.589902+0000","last_became_peered":"2024-09-15T23:36:23.589902+0000","last_unstale":"2024-09-15T23:36:28.622706+0000","last_undegraded":"2024-09-15T23:36:28.622706+0000","last_fullsized":"2024-09-15T23:36:28.622706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:37.196308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024145+0000","last_change":"2024-09-15T23:36:27.603610+0000","last_active":"2024-09-15T23:36:30.024145+0000","last_peered":"2024-09-15T23:36:30.024145+0000","last_clean":"2024-09-15T23:36:30.024145+0000","last_became_active":"2024-09-15T23:36:27.603177+0000","last_became_peered":"2024-09-15T23:36:27.603177+0000","last_unstale":"2024-09-15T23:36:30.024145+0000","last_undegraded":"2024-09-15T23:36:30.024145+0000","last_fullsized":"2024-09-15T23:36:30.024145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:55.070642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622285+0000","last_change":"2024-09-15T23:36:23.576248+0000","last_active":"2024-09-15T23:36:28.622285+0000","last_peered":"2024-09-15T23:36:28.622285+0000","last_clean":"2024-09-15T23:36:28.622285+0000","last_became_active":"2024-09-15T23:36:23.575715+0000","last_became_peered":"2024-09-15T23:36:23.575715+0000","last_unstale":"2024-09-15T23:36:28.622285+0000","last_undegraded":"2024-09-15T23:36:28.622285+0000","last_fullsized":"2024-09-15T23:36:28.622285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:59.081263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622550+0000","last_change":"2024-09-15T23:36:22.555862+0000","last_active":"2024-09-15T23:36:28.622550+0000","last_peered":"2024-09-15T23:36:28.622550+0000","last_clean":"2024-09-15T23:36:28.622550+0000","last_became_active":"2024-09-15T23:36:22.555589+0000","last_became_peered":"2024-09-15T23:36:22.555589+0000","last_unstale":"2024-09-15T23:36:28.622550+0000","last_undegraded":"2024-09-15T23:36:28.622550+0000","last_fullsized":"2024-09-15T23:36:28.622550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:44.229130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622578+0000","last_change":"2024-09-15T23:36:27.602404+0000","last_active":"2024-09-15T23:36:28.622578+0000","last_peered":"2024-09-15T23:36:28.622578+0000","last_clean":"2024-09-15T23:36:28.622578+0000","last_became_active":"2024-09-15T23:36:27.601991+0000","last_became_peered":"2024-09-15T23:36:27.601991+0000","last_unstale":"2024-09-15T23:36:28.622578+0000","last_undegraded":"2024-09-15T23:36:28.622578+0000","last_fullsized":"2024-09-15T23:36:28.622578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:00.662926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454749+0000","last_change":"2024-09-15T23:36:23.591252+0000","last_active":"2024-09-15T23:36:29.454749+0000","last_peered":"2024-09-15T23:36:29.454749+0000","last_clean":"2024-09-15T23:36:29.454749+0000","last_became_active":"2024-09-15T23:36:23.590976+0000","last_became_peered":"2024-09-15T23:36:23.590976+0000","last_unstale":"2024-09-15T23:36:29.454749+0000","last_undegraded":"2024-09-15T23:36:29.454749+0000","last_fullsized":"2024-09-15T23:36:29.454749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622161+0000","last_change":"2024-09-15T23:36:22.560352+0000","last_active":"2024-09-15T23:36:28.622161+0000","last_peered":"2024-09-15T23:36:28.622161+0000","last_clean":"2024-09-15T23:36:28.622161+0000","last_became_active":"2024-09-15T23:36:22.560052+0000","last_became_peered":"2024-09-15T23:36:22.560052+0000","last_unstale":"2024-09-15T23:36:28.622161+0000","last_undegraded":"2024-09-15T23:36:28.622161+0000","last_fullsized":"2024-09-15T23:36:28.622161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023452+0000","last_change":"2024-09-15T23:36:27.598358+0000","last_active":"2024-09-15T23:36:30.023452+0000","last_peered":"2024-09-15T23:36:30.023452+0000","last_clean":"2024-09-15T23:36:30.023452+0000","last_became_active":"2024-09-15T23:36:27.597941+0000","last_became_peered":"2024-09-15T23:36:27.597941+0000","last_unstale":"2024-09-15T23:36:30.023452+0000","last_undegraded":"2024-09-15T23:36:30.023452+0000","last_fullsized":"2024-09-15T23:36:30.023452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:54.115527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622271+0000","last_change":"2024-09-15T23:36:23.581091+0000","last_active":"2024-09-15T23:36:28.622271+0000","last_peered":"2024-09-15T23:36:28.622271+0000","last_clean":"2024-09-15T23:36:28.622271+0000","last_became_active":"2024-09-15T23:36:23.580807+0000","last_became_peered":"2024-09-15T23:36:23.580807+0000","last_unstale":"2024-09-15T23:36:28.622271+0000","last_undegraded":"2024-09-15T23:36:28.622271+0000","last_fullsized":"2024-09-15T23:36:28.622271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:01.465502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623733+0000","last_change":"2024-09-15T23:36:22.569473+0000","last_active":"2024-09-15T23:36:28.623733+0000","last_peered":"2024-09-15T23:36:28.623733+0000","last_clean":"2024-09-15T23:36:28.623733+0000","last_became_active":"2024-09-15T23:36:22.569168+0000","last_became_peered":"2024-09-15T23:36:22.569168+0000","last_unstale":"2024-09-15T23:36:28.623733+0000","last_undegraded":"2024-09-15T23:36:28.623733+0000","last_fullsized":"2024-09-15T23:36:28.623733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:46.065174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454933+0000","last_change":"2024-09-15T23:36:27.605801+0000","last_active":"2024-09-15T23:36:29.454933+0000","last_peered":"2024-09-15T23:36:29.454933+0000","last_clean":"2024-09-15T23:36:29.454933+0000","last_became_active":"2024-09-15T23:36:27.605416+0000","last_became_peered":"2024-09-15T23:36:27.605416+0000","last_unstale":"2024-09-15T23:36:29.454933+0000","last_undegraded":"2024-09-15T23:36:29.454933+0000","last_fullsized":"2024-09-15T23:36:29.454933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:27.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622056+0000","last_change":"2024-09-15T23:36:23.587550+0000","last_active":"2024-09-15T23:36:28.622056+0000","last_peered":"2024-09-15T23:36:28.622056+0000","last_clean":"2024-09-15T23:36:28.622056+0000","last_became_active":"2024-09-15T23:36:23.587213+0000","last_became_peered":"2024-09-15T23:36:23.587213+0000","last_unstale":"2024-09-15T23:36:28.622056+0000","last_undegraded":"2024-09-15T23:36:28.622056+0000","last_fullsized":"2024-09-15T23:36:28.622056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:15.353430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023245+0000","last_change":"2024-09-15T23:36:22.569502+0000","last_active":"2024-09-15T23:36:30.023245+0000","last_peered":"2024-09-15T23:36:30.023245+0000","last_clean":"2024-09-15T23:36:30.023245+0000","last_became_active":"2024-09-15T23:36:22.567715+0000","last_became_peered":"2024-09-15T23:36:22.567715+0000","last_unstale":"2024-09-15T23:36:30.023245+0000","last_undegraded":"2024-09-15T23:36:30.023245+0000","last_fullsized":"2024-09-15T23:36:30.023245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024741+0000","last_change":"2024-09-15T23:36:27.612217+0000","last_active":"2024-09-15T23:36:30.024741+0000","last_peered":"2024-09-15T23:36:30.024741+0000","last_clean":"2024-09-15T23:36:30.024741+0000","last_became_active":"2024-09-15T23:36:27.611783+0000","last_became_peered":"2024-09-15T23:36:27.611783+0000","last_unstale":"2024-09-15T23:36:30.024741+0000","last_undegraded":"2024-09-15T23:36:30.024741+0000","last_fullsized":"2024-09-15T23:36:30.024741+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:45.020968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623852+0000","last_change":"2024-09-15T23:36:23.576178+0000","last_active":"2024-09-15T23:36:28.623852+0000","last_peered":"2024-09-15T23:36:28.623852+0000","last_clean":"2024-09-15T23:36:28.623852+0000","last_became_active":"2024-09-15T23:36:23.575823+0000","last_became_peered":"2024-09-15T23:36:23.575823+0000","last_unstale":"2024-09-15T23:36:28.623852+0000","last_undegraded":"2024-09-15T23:36:28.623852+0000","last_fullsized":"2024-09-15T23:36:28.623852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:17.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623244+0000","last_change":"2024-09-15T23:36:22.570936+0000","last_active":"2024-09-15T23:36:28.623244+0000","last_peered":"2024-09-15T23:36:28.623244+0000","last_clean":"2024-09-15T23:36:28.623244+0000","last_became_active":"2024-09-15T23:36:22.569461+0000","last_became_peered":"2024-09-15T23:36:22.569461+0000","last_unstale":"2024-09-15T23:36:28.623244+0000","last_undegraded":"2024-09-15T23:36:28.623244+0000","last_fullsized":"2024-09-15T23:36:28.623244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:38.263694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024707+0000","last_change":"2024-09-15T23:36:27.605878+0000","last_active":"2024-09-15T23:36:30.024707+0000","last_peered":"2024-09-15T23:36:30.024707+0000","last_clean":"2024-09-15T23:36:30.024707+0000","last_became_active":"2024-09-15T23:36:27.605448+0000","last_became_peered":"2024-09-15T23:36:27.605448+0000","last_unstale":"2024-09-15T23:36:30.024707+0000","last_undegraded":"2024-09-15T23:36:30.024707+0000","last_fullsized":"2024-09-15T23:36:30.024707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:00.293850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623345+0000","last_change":"2024-09-15T23:36:23.590802+0000","last_active":"2024-09-15T23:36:28.623345+0000","last_peered":"2024-09-15T23:36:28.623345+0000","last_clean":"2024-09-15T23:36:28.623345+0000","last_became_active":"2024-09-15T23:36:23.590505+0000","last_became_peered":"2024-09-15T23:36:23.590505+0000","last_unstale":"2024-09-15T23:36:28.623345+0000","last_undegraded":"2024-09-15T23:36:28.623345+0000","last_fullsized":"2024-09-15T23:36:28.623345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:50.950584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624097+0000","last_change":"2024-09-15T23:36:22.569675+0000","last_active":"2024-09-15T23:36:28.624097+0000","last_peered":"2024-09-15T23:36:28.624097+0000","last_clean":"2024-09-15T23:36:28.624097+0000","last_became_active":"2024-09-15T23:36:22.569386+0000","last_became_peered":"2024-09-15T23:36:22.569386+0000","last_unstale":"2024-09-15T23:36:28.624097+0000","last_undegraded":"2024-09-15T23:36:28.624097+0000","last_fullsized":"2024-09-15T23:36:28.624097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:00.420124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622250+0000","last_change":"2024-09-15T23:36:27.601084+0000","last_active":"2024-09-15T23:36:28.622250+0000","last_peered":"2024-09-15T23:36:28.622250+0000","last_clean":"2024-09-15T23:36:28.622250+0000","last_became_active":"2024-09-15T23:36:27.600671+0000","last_became_peered":"2024-09-15T23:36:27.600671+0000","last_unstale":"2024-09-15T23:36:28.622250+0000","last_undegraded":"2024-09-15T23:36:28.622250+0000","last_fullsized":"2024-09-15T23:36:28.622250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:28.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622694+0000","last_change":"2024-09-15T23:36:23.595285+0000","last_active":"2024-09-15T23:36:28.622694+0000","last_peered":"2024-09-15T23:36:28.622694+0000","last_clean":"2024-09-15T23:36:28.622694+0000","last_became_active":"2024-09-15T23:36:23.594596+0000","last_became_peered":"2024-09-15T23:36:23.594596+0000","last_unstale":"2024-09-15T23:36:28.622694+0000","last_undegraded":"2024-09-15T23:36:28.622694+0000","last_fullsized":"2024-09-15T23:36:28.622694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:39.265619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623245+0000","last_change":"2024-09-15T23:36:22.553052+0000","last_active":"2024-09-15T23:36:28.623245+0000","last_peered":"2024-09-15T23:36:28.623245+0000","last_clean":"2024-09-15T23:36:28.623245+0000","last_became_active":"2024-09-15T23:36:22.552766+0000","last_became_peered":"2024-09-15T23:36:22.552766+0000","last_unstale":"2024-09-15T23:36:28.623245+0000","last_undegraded":"2024-09-15T23:36:28.623245+0000","last_fullsized":"2024-09-15T23:36:28.623245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:45.928194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621909+0000","last_change":"2024-09-15T23:36:27.601501+0000","last_active":"2024-09-15T23:36:28.621909+0000","last_peered":"2024-09-15T23:36:28.621909+0000","last_clean":"2024-09-15T23:36:28.621909+0000","last_became_active":"2024-09-15T23:36:27.601071+0000","last_became_peered":"2024-09-15T23:36:27.601071+0000","last_unstale":"2024-09-15T23:36:28.621909+0000","last_undegraded":"2024-09-15T23:36:28.621909+0000","last_fullsized":"2024-09-15T23:36:28.621909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:03.241943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024193+0000","last_change":"2024-09-15T23:36:23.591415+0000","last_active":"2024-09-15T23:36:30.024193+0000","last_peered":"2024-09-15T23:36:30.024193+0000","last_clean":"2024-09-15T23:36:30.024193+0000","last_became_active":"2024-09-15T23:36:23.591132+0000","last_became_peered":"2024-09-15T23:36:23.591132+0000","last_unstale":"2024-09-15T23:36:30.024193+0000","last_undegraded":"2024-09-15T23:36:30.024193+0000","last_fullsized":"2024-09-15T23:36:30.024193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024034+0000","last_change":"2024-09-15T23:36:22.561640+0000","last_active":"2024-09-15T23:36:30.024034+0000","last_peered":"2024-09-15T23:36:30.024034+0000","last_clean":"2024-09-15T23:36:30.024034+0000","last_became_active":"2024-09-15T23:36:22.561218+0000","last_became_peered":"2024-09-15T23:36:22.561218+0000","last_unstale":"2024-09-15T23:36:30.024034+0000","last_undegraded":"2024-09-15T23:36:30.024034+0000","last_fullsized":"2024-09-15T23:36:30.024034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:06:31.226510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621863+0000","last_change":"2024-09-15T23:36:27.606889+0000","last_active":"2024-09-15T23:36:28.621863+0000","last_peered":"2024-09-15T23:36:28.621863+0000","last_clean":"2024-09-15T23:36:28.621863+0000","last_became_active":"2024-09-15T23:36:27.606466+0000","last_became_peered":"2024-09-15T23:36:27.606466+0000","last_unstale":"2024-09-15T23:36:28.621863+0000","last_undegraded":"2024-09-15T23:36:28.621863+0000","last_fullsized":"2024-09-15T23:36:28.621863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:55.070642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023954+0000","last_change":"2024-09-15T23:36:23.596605+0000","last_active":"2024-09-15T23:36:30.023954+0000","last_peered":"2024-09-15T23:36:30.023954+0000","last_clean":"2024-09-15T23:36:30.023954+0000","last_became_active":"2024-09-15T23:36:23.596318+0000","last_became_peered":"2024-09-15T23:36:23.596318+0000","last_unstale":"2024-09-15T23:36:30.023954+0000","last_undegraded":"2024-09-15T23:36:30.023954+0000","last_fullsized":"2024-09-15T23:36:30.023954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:02.746761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622853+0000","last_change":"2024-09-15T23:36:22.559703+0000","last_active":"2024-09-15T23:36:28.622853+0000","last_peered":"2024-09-15T23:36:28.622853+0000","last_clean":"2024-09-15T23:36:28.622853+0000","last_became_active":"2024-09-15T23:36:22.559271+0000","last_became_peered":"2024-09-15T23:36:22.559271+0000","last_unstale":"2024-09-15T23:36:28.622853+0000","last_undegraded":"2024-09-15T23:36:28.622853+0000","last_fullsized":"2024-09-15T23:36:28.622853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:23.347088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622571+0000","last_change":"2024-09-15T23:36:27.598188+0000","last_active":"2024-09-15T23:36:28.622571+0000","last_peered":"2024-09-15T23:36:28.622571+0000","last_clean":"2024-09-15T23:36:28.622571+0000","last_became_active":"2024-09-15T23:36:27.596219+0000","last_became_peered":"2024-09-15T23:36:27.596219+0000","last_unstale":"2024-09-15T23:36:28.622571+0000","last_undegraded":"2024-09-15T23:36:28.622571+0000","last_fullsized":"2024-09-15T23:36:28.622571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:23.776974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023619+0000","last_change":"2024-09-15T23:36:23.582672+0000","last_active":"2024-09-15T23:36:30.023619+0000","last_peered":"2024-09-15T23:36:30.023619+0000","last_clean":"2024-09-15T23:36:30.023619+0000","last_became_active":"2024-09-15T23:36:23.582388+0000","last_became_peered":"2024-09-15T23:36:23.582388+0000","last_unstale":"2024-09-15T23:36:30.023619+0000","last_undegraded":"2024-09-15T23:36:30.023619+0000","last_fullsized":"2024-09-15T23:36:30.023619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:07.005224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454431+0000","last_change":"2024-09-15T23:36:22.561458+0000","last_active":"2024-09-15T23:36:29.454431+0000","last_peered":"2024-09-15T23:36:29.454431+0000","last_clean":"2024-09-15T23:36:29.454431+0000","last_became_active":"2024-09-15T23:36:22.561171+0000","last_became_peered":"2024-09-15T23:36:22.561171+0000","last_unstale":"2024-09-15T23:36:29.454431+0000","last_undegraded":"2024-09-15T23:36:29.454431+0000","last_fullsized":"2024-09-15T23:36:29.454431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622564+0000","last_change":"2024-09-15T23:36:27.597617+0000","last_active":"2024-09-15T23:36:28.622564+0000","last_peered":"2024-09-15T23:36:28.622564+0000","last_clean":"2024-09-15T23:36:28.622564+0000","last_became_active":"2024-09-15T23:36:27.597224+0000","last_became_peered":"2024-09-15T23:36:27.597224+0000","last_unstale":"2024-09-15T23:36:28.622564+0000","last_undegraded":"2024-09-15T23:36:28.622564+0000","last_fullsized":"2024-09-15T23:36:28.622564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:06.783968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621714+0000","last_change":"2024-09-15T23:36:23.590567+0000","last_active":"2024-09-15T23:36:28.621714+0000","last_peered":"2024-09-15T23:36:28.621714+0000","last_clean":"2024-09-15T23:36:28.621714+0000","last_became_active":"2024-09-15T23:36:23.590272+0000","last_became_peered":"2024-09-15T23:36:23.590272+0000","last_unstale":"2024-09-15T23:36:28.621714+0000","last_undegraded":"2024-09-15T23:36:28.621714+0000","last_fullsized":"2024-09-15T23:36:28.621714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622169+0000","last_change":"2024-09-15T23:36:22.560601+0000","last_active":"2024-09-15T23:36:28.622169+0000","last_peered":"2024-09-15T23:36:28.622169+0000","last_clean":"2024-09-15T23:36:28.622169+0000","last_became_active":"2024-09-15T23:36:22.560306+0000","last_became_peered":"2024-09-15T23:36:22.560306+0000","last_unstale":"2024-09-15T23:36:28.622169+0000","last_undegraded":"2024-09-15T23:36:28.622169+0000","last_fullsized":"2024-09-15T23:36:28.622169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:38.283320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024190+0000","last_change":"2024-09-15T23:36:27.610042+0000","last_active":"2024-09-15T23:36:30.024190+0000","last_peered":"2024-09-15T23:36:30.024190+0000","last_clean":"2024-09-15T23:36:30.024190+0000","last_became_active":"2024-09-15T23:36:27.609630+0000","last_became_peered":"2024-09-15T23:36:27.609630+0000","last_unstale":"2024-09-15T23:36:30.024190+0000","last_undegraded":"2024-09-15T23:36:30.024190+0000","last_fullsized":"2024-09-15T23:36:30.024190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:03.827160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624171+0000","last_change":"2024-09-15T23:36:23.580635+0000","last_active":"2024-09-15T23:36:28.624171+0000","last_peered":"2024-09-15T23:36:28.624171+0000","last_clean":"2024-09-15T23:36:28.624171+0000","last_became_active":"2024-09-15T23:36:23.580348+0000","last_became_peered":"2024-09-15T23:36:23.580348+0000","last_unstale":"2024-09-15T23:36:28.624171+0000","last_undegraded":"2024-09-15T23:36:28.624171+0000","last_fullsized":"2024-09-15T23:36:28.624171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:03.646324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622420+0000","last_change":"2024-09-15T23:36:22.552162+0000","last_active":"2024-09-15T23:36:28.622420+0000","last_peered":"2024-09-15T23:36:28.622420+0000","last_clean":"2024-09-15T23:36:28.622420+0000","last_became_active":"2024-09-15T23:36:22.551788+0000","last_became_peered":"2024-09-15T23:36:22.551788+0000","last_unstale":"2024-09-15T23:36:28.622420+0000","last_undegraded":"2024-09-15T23:36:28.622420+0000","last_fullsized":"2024-09-15T23:36:28.622420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:04.332504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023778+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:36:30.023778+0000","last_peered":"2024-09-15T23:36:30.023778+0000","last_clean":"2024-09-15T23:36:30.023778+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:36:30.023778+0000","last_undegraded":"2024-09-15T23:36:30.023778+0000","last_fullsized":"2024-09-15T23:36:30.023778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623010+0000","last_change":"2024-09-15T23:36:27.604371+0000","last_active":"2024-09-15T23:36:28.623010+0000","last_peered":"2024-09-15T23:36:28.623010+0000","last_clean":"2024-09-15T23:36:28.623010+0000","last_became_active":"2024-09-15T23:36:27.603926+0000","last_became_peered":"2024-09-15T23:36:27.603926+0000","last_unstale":"2024-09-15T23:36:28.623010+0000","last_undegraded":"2024-09-15T23:36:28.623010+0000","last_fullsized":"2024-09-15T23:36:28.623010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:41.910981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622897+0000","last_change":"2024-09-15T23:36:23.575297+0000","last_active":"2024-09-15T23:36:28.622897+0000","last_peered":"2024-09-15T23:36:28.622897+0000","last_clean":"2024-09-15T23:36:28.622897+0000","last_became_active":"2024-09-15T23:36:23.574997+0000","last_became_peered":"2024-09-15T23:36:23.574997+0000","last_unstale":"2024-09-15T23:36:28.622897+0000","last_undegraded":"2024-09-15T23:36:28.622897+0000","last_fullsized":"2024-09-15T23:36:28.622897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:24.495389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622829+0000","last_change":"2024-09-15T23:36:22.569366+0000","last_active":"2024-09-15T23:36:28.622829+0000","last_peered":"2024-09-15T23:36:28.622829+0000","last_clean":"2024-09-15T23:36:28.622829+0000","last_became_active":"2024-09-15T23:36:22.569043+0000","last_became_peered":"2024-09-15T23:36:22.569043+0000","last_unstale":"2024-09-15T23:36:28.622829+0000","last_undegraded":"2024-09-15T23:36:28.622829+0000","last_fullsized":"2024-09-15T23:36:28.622829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:55.907462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024315+0000","last_change":"2024-09-15T23:36:27.611141+0000","last_active":"2024-09-15T23:36:30.024315+0000","last_peered":"2024-09-15T23:36:30.024315+0000","last_clean":"2024-09-15T23:36:30.024315+0000","last_became_active":"2024-09-15T23:36:27.610654+0000","last_became_peered":"2024-09-15T23:36:27.610654+0000","last_unstale":"2024-09-15T23:36:30.024315+0000","last_undegraded":"2024-09-15T23:36:30.024315+0000","last_fullsized":"2024-09-15T23:36:30.024315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:03.241943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025277+0000","last_change":"2024-09-15T23:36:23.592044+0000","last_active":"2024-09-15T23:36:30.025277+0000","last_peered":"2024-09-15T23:36:30.025277+0000","last_clean":"2024-09-15T23:36:30.025277+0000","last_became_active":"2024-09-15T23:36:23.591419+0000","last_became_peered":"2024-09-15T23:36:23.591419+0000","last_unstale":"2024-09-15T23:36:30.025277+0000","last_undegraded":"2024-09-15T23:36:30.025277+0000","last_fullsized":"2024-09-15T23:36:30.025277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:59.204736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024210+0000","last_change":"2024-09-15T23:36:22.568129+0000","last_active":"2024-09-15T23:36:30.024210+0000","last_peered":"2024-09-15T23:36:30.024210+0000","last_clean":"2024-09-15T23:36:30.024210+0000","last_became_active":"2024-09-15T23:36:22.567909+0000","last_became_peered":"2024-09-15T23:36:22.567909+0000","last_unstale":"2024-09-15T23:36:30.024210+0000","last_undegraded":"2024-09-15T23:36:30.024210+0000","last_fullsized":"2024-09-15T23:36:30.024210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:46.065174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024320+0000","last_change":"2024-09-15T23:36:27.610796+0000","last_active":"2024-09-15T23:36:30.024320+0000","last_peered":"2024-09-15T23:36:30.024320+0000","last_clean":"2024-09-15T23:36:30.024320+0000","last_became_active":"2024-09-15T23:36:27.610355+0000","last_became_peered":"2024-09-15T23:36:27.610355+0000","last_unstale":"2024-09-15T23:36:30.024320+0000","last_undegraded":"2024-09-15T23:36:30.024320+0000","last_fullsized":"2024-09-15T23:36:30.024320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:22.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024449+0000","last_change":"2024-09-15T23:36:23.592411+0000","last_active":"2024-09-15T23:36:30.024449+0000","last_peered":"2024-09-15T23:36:30.024449+0000","last_clean":"2024-09-15T23:36:30.024449+0000","last_became_active":"2024-09-15T23:36:23.592130+0000","last_became_peered":"2024-09-15T23:36:23.592130+0000","last_unstale":"2024-09-15T23:36:30.024449+0000","last_undegraded":"2024-09-15T23:36:30.024449+0000","last_fullsized":"2024-09-15T23:36:30.024449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.872038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623974+0000","last_change":"2024-09-15T23:36:22.560112+0000","last_active":"2024-09-15T23:36:28.623974+0000","last_peered":"2024-09-15T23:36:28.623974+0000","last_clean":"2024-09-15T23:36:28.623974+0000","last_became_active":"2024-09-15T23:36:22.559811+0000","last_became_peered":"2024-09-15T23:36:22.559811+0000","last_unstale":"2024-09-15T23:36:28.623974+0000","last_undegraded":"2024-09-15T23:36:28.623974+0000","last_fullsized":"2024-09-15T23:36:28.623974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:46.482072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621872+0000","last_change":"2024-09-15T23:36:27.603263+0000","last_active":"2024-09-15T23:36:28.621872+0000","last_peered":"2024-09-15T23:36:28.621872+0000","last_clean":"2024-09-15T23:36:28.621872+0000","last_became_active":"2024-09-15T23:36:27.602856+0000","last_became_peered":"2024-09-15T23:36:27.602856+0000","last_unstale":"2024-09-15T23:36:28.621872+0000","last_undegraded":"2024-09-15T23:36:28.621872+0000","last_fullsized":"2024-09-15T23:36:28.621872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:38.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024861+0000","last_change":"2024-09-15T23:36:23.594876+0000","last_active":"2024-09-15T23:36:30.024861+0000","last_peered":"2024-09-15T23:36:30.024861+0000","last_clean":"2024-09-15T23:36:30.024861+0000","last_became_active":"2024-09-15T23:36:23.594592+0000","last_became_peered":"2024-09-15T23:36:23.594592+0000","last_unstale":"2024-09-15T23:36:30.024861+0000","last_undegraded":"2024-09-15T23:36:30.024861+0000","last_fullsized":"2024-09-15T23:36:30.024861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:05.334429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024659+0000","last_change":"2024-09-15T23:36:22.568643+0000","last_active":"2024-09-15T23:36:30.024659+0000","last_peered":"2024-09-15T23:36:30.024659+0000","last_clean":"2024-09-15T23:36:30.024659+0000","last_became_active":"2024-09-15T23:36:22.568351+0000","last_became_peered":"2024-09-15T23:36:22.568351+0000","last_unstale":"2024-09-15T23:36:30.024659+0000","last_undegraded":"2024-09-15T23:36:30.024659+0000","last_fullsized":"2024-09-15T23:36:30.024659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:39.192475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023472+0000","last_change":"2024-09-15T23:36:27.605261+0000","last_active":"2024-09-15T23:36:30.023472+0000","last_peered":"2024-09-15T23:36:30.023472+0000","last_clean":"2024-09-15T23:36:30.023472+0000","last_became_active":"2024-09-15T23:36:27.604780+0000","last_became_peered":"2024-09-15T23:36:27.604780+0000","last_unstale":"2024-09-15T23:36:30.023472+0000","last_undegraded":"2024-09-15T23:36:30.023472+0000","last_fullsized":"2024-09-15T23:36:30.023472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:09.371636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621977+0000","last_change":"2024-09-15T23:36:23.582678+0000","last_active":"2024-09-15T23:36:28.621977+0000","last_peered":"2024-09-15T23:36:28.621977+0000","last_clean":"2024-09-15T23:36:28.621977+0000","last_became_active":"2024-09-15T23:36:23.582022+0000","last_became_peered":"2024-09-15T23:36:23.582022+0000","last_unstale":"2024-09-15T23:36:28.621977+0000","last_undegraded":"2024-09-15T23:36:28.621977+0000","last_fullsized":"2024-09-15T23:36:28.621977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:14.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622315+0000","last_change":"2024-09-15T23:36:22.552625+0000","last_active":"2024-09-15T23:36:28.622315+0000","last_peered":"2024-09-15T23:36:28.622315+0000","last_clean":"2024-09-15T23:36:28.622315+0000","last_became_active":"2024-09-15T23:36:22.552330+0000","last_became_peered":"2024-09-15T23:36:22.552330+0000","last_unstale":"2024-09-15T23:36:28.622315+0000","last_undegraded":"2024-09-15T23:36:28.622315+0000","last_fullsized":"2024-09-15T23:36:28.622315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:06:31.226510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024467+0000","last_change":"2024-09-15T23:36:27.607752+0000","last_active":"2024-09-15T23:36:30.024467+0000","last_peered":"2024-09-15T23:36:30.024467+0000","last_clean":"2024-09-15T23:36:30.024467+0000","last_became_active":"2024-09-15T23:36:27.607269+0000","last_became_peered":"2024-09-15T23:36:27.607269+0000","last_unstale":"2024-09-15T23:36:30.024467+0000","last_undegraded":"2024-09-15T23:36:30.024467+0000","last_fullsized":"2024-09-15T23:36:30.024467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:55.005028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023687+0000","last_change":"2024-09-15T23:36:23.585722+0000","last_active":"2024-09-15T23:36:30.023687+0000","last_peered":"2024-09-15T23:36:30.023687+0000","last_clean":"2024-09-15T23:36:30.023687+0000","last_became_active":"2024-09-15T23:36:23.585435+0000","last_became_peered":"2024-09-15T23:36:23.585435+0000","last_unstale":"2024-09-15T23:36:30.023687+0000","last_undegraded":"2024-09-15T23:36:30.023687+0000","last_fullsized":"2024-09-15T23:36:30.023687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023646+0000","last_change":"2024-09-15T23:36:22.568106+0000","last_active":"2024-09-15T23:36:30.023646+0000","last_peered":"2024-09-15T23:36:30.023646+0000","last_clean":"2024-09-15T23:36:30.023646+0000","last_became_active":"2024-09-15T23:36:22.567770+0000","last_became_peered":"2024-09-15T23:36:22.567770+0000","last_unstale":"2024-09-15T23:36:30.023646+0000","last_undegraded":"2024-09-15T23:36:30.023646+0000","last_fullsized":"2024-09-15T23:36:30.023646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:57.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024431+0000","last_change":"2024-09-15T23:36:27.611234+0000","last_active":"2024-09-15T23:36:30.024431+0000","last_peered":"2024-09-15T23:36:30.024431+0000","last_clean":"2024-09-15T23:36:30.024431+0000","last_became_active":"2024-09-15T23:36:27.610828+0000","last_became_peered":"2024-09-15T23:36:27.610828+0000","last_unstale":"2024-09-15T23:36:30.024431+0000","last_undegraded":"2024-09-15T23:36:30.024431+0000","last_fullsized":"2024-09-15T23:36:30.024431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:30.372516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622286+0000","last_change":"2024-09-15T23:36:23.592816+0000","last_active":"2024-09-15T23:36:28.622286+0000","last_peered":"2024-09-15T23:36:28.622286+0000","last_clean":"2024-09-15T23:36:28.622286+0000","last_became_active":"2024-09-15T23:36:23.590219+0000","last_became_peered":"2024-09-15T23:36:23.590219+0000","last_unstale":"2024-09-15T23:36:28.622286+0000","last_undegraded":"2024-09-15T23:36:28.622286+0000","last_fullsized":"2024-09-15T23:36:28.622286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:09:38.065436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622572+0000","last_change":"2024-09-15T23:36:22.555612+0000","last_active":"2024-09-15T23:36:28.622572+0000","last_peered":"2024-09-15T23:36:28.622572+0000","last_clean":"2024-09-15T23:36:28.622572+0000","last_became_active":"2024-09-15T23:36:22.555279+0000","last_became_peered":"2024-09-15T23:36:22.555279+0000","last_unstale":"2024-09-15T23:36:28.622572+0000","last_undegraded":"2024-09-15T23:36:28.622572+0000","last_fullsized":"2024-09-15T23:36:28.622572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:18.004944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622407+0000","last_change":"2024-09-15T23:36:27.594766+0000","last_active":"2024-09-15T23:36:28.622407+0000","last_peered":"2024-09-15T23:36:28.622407+0000","last_clean":"2024-09-15T23:36:28.622407+0000","last_became_active":"2024-09-15T23:36:27.594179+0000","last_became_peered":"2024-09-15T23:36:27.594179+0000","last_unstale":"2024-09-15T23:36:28.622407+0000","last_undegraded":"2024-09-15T23:36:28.622407+0000","last_fullsized":"2024-09-15T23:36:28.622407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:43.647541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622052+0000","last_change":"2024-09-15T23:36:23.595059+0000","last_active":"2024-09-15T23:36:28.622052+0000","last_peered":"2024-09-15T23:36:28.622052+0000","last_clean":"2024-09-15T23:36:28.622052+0000","last_became_active":"2024-09-15T23:36:23.594760+0000","last_became_peered":"2024-09-15T23:36:23.594760+0000","last_unstale":"2024-09-15T23:36:28.622052+0000","last_undegraded":"2024-09-15T23:36:28.622052+0000","last_fullsized":"2024-09-15T23:36:28.622052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:20.169043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454742+0000","last_change":"2024-09-15T23:36:22.569556+0000","last_active":"2024-09-15T23:36:29.454742+0000","last_peered":"2024-09-15T23:36:29.454742+0000","last_clean":"2024-09-15T23:36:29.454742+0000","last_became_active":"2024-09-15T23:36:22.569214+0000","last_became_peered":"2024-09-15T23:36:22.569214+0000","last_unstale":"2024-09-15T23:36:29.454742+0000","last_undegraded":"2024-09-15T23:36:29.454742+0000","last_fullsized":"2024-09-15T23:36:29.454742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:14:47.610942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622749+0000","last_change":"2024-09-15T23:36:27.601370+0000","last_active":"2024-09-15T23:36:28.622749+0000","last_peered":"2024-09-15T23:36:28.622749+0000","last_clean":"2024-09-15T23:36:28.622749+0000","last_became_active":"2024-09-15T23:36:27.600995+0000","last_became_peered":"2024-09-15T23:36:27.600995+0000","last_unstale":"2024-09-15T23:36:28.622749+0000","last_undegraded":"2024-09-15T23:36:28.622749+0000","last_fullsized":"2024-09-15T23:36:28.622749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:13.579126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025128+0000","last_change":"2024-09-15T23:36:23.595877+0000","last_active":"2024-09-15T23:36:30.025128+0000","last_peered":"2024-09-15T23:36:30.025128+0000","last_clean":"2024-09-15T23:36:30.025128+0000","last_became_active":"2024-09-15T23:36:23.595086+0000","last_became_peered":"2024-09-15T23:36:23.595086+0000","last_unstale":"2024-09-15T23:36:30.025128+0000","last_undegraded":"2024-09-15T23:36:30.025128+0000","last_fullsized":"2024-09-15T23:36:30.025128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:19.739767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:33.124757+0000","last_change":"2024-09-15T23:36:22.551636+0000","last_active":"2024-09-15T23:36:33.124757+0000","last_peered":"2024-09-15T23:36:33.124757+0000","last_clean":"2024-09-15T23:36:33.124757+0000","last_became_active":"2024-09-15T23:36:22.551316+0000","last_became_peered":"2024-09-15T23:36:22.551316+0000","last_unstale":"2024-09-15T23:36:33.124757+0000","last_undegraded":"2024-09-15T23:36:33.124757+0000","last_fullsized":"2024-09-15T23:36:33.124757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624158+0000","last_change":"2024-09-15T23:36:27.606629+0000","last_active":"2024-09-15T23:36:28.624158+0000","last_peered":"2024-09-15T23:36:28.624158+0000","last_clean":"2024-09-15T23:36:28.624158+0000","last_became_active":"2024-09-15T23:36:27.606224+0000","last_became_peered":"2024-09-15T23:36:27.606224+0000","last_unstale":"2024-09-15T23:36:28.624158+0000","last_undegraded":"2024-09-15T23:36:28.624158+0000","last_fullsized":"2024-09-15T23:36:28.624158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:09.576731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622435+0000","last_change":"2024-09-15T23:36:23.580904+0000","last_active":"2024-09-15T23:36:28.622435+0000","last_peered":"2024-09-15T23:36:28.622435+0000","last_clean":"2024-09-15T23:36:28.622435+0000","last_became_active":"2024-09-15T23:36:23.580459+0000","last_became_peered":"2024-09-15T23:36:23.580459+0000","last_unstale":"2024-09-15T23:36:28.622435+0000","last_undegraded":"2024-09-15T23:36:28.622435+0000","last_fullsized":"2024-09-15T23:36:28.622435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:50.078320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454752+0000","last_change":"2024-09-15T23:36:22.557264+0000","last_active":"2024-09-15T23:36:29.454752+0000","last_peered":"2024-09-15T23:36:29.454752+0000","last_clean":"2024-09-15T23:36:29.454752+0000","last_became_active":"2024-09-15T23:36:22.556929+0000","last_became_peered":"2024-09-15T23:36:22.556929+0000","last_unstale":"2024-09-15T23:36:29.454752+0000","last_undegraded":"2024-09-15T23:36:29.454752+0000","last_fullsized":"2024-09-15T23:36:29.454752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:57.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024486+0000","last_change":"2024-09-15T23:36:27.605955+0000","last_active":"2024-09-15T23:36:30.024486+0000","last_peered":"2024-09-15T23:36:30.024486+0000","last_clean":"2024-09-15T23:36:30.024486+0000","last_became_active":"2024-09-15T23:36:27.605521+0000","last_became_peered":"2024-09-15T23:36:27.605521+0000","last_unstale":"2024-09-15T23:36:30.024486+0000","last_undegraded":"2024-09-15T23:36:30.024486+0000","last_fullsized":"2024-09-15T23:36:30.024486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:23.776974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024511+0000","last_change":"2024-09-15T23:36:23.583781+0000","last_active":"2024-09-15T23:36:30.024511+0000","last_peered":"2024-09-15T23:36:30.024511+0000","last_clean":"2024-09-15T23:36:30.024511+0000","last_became_active":"2024-09-15T23:36:23.583475+0000","last_became_peered":"2024-09-15T23:36:23.583475+0000","last_unstale":"2024-09-15T23:36:30.024511+0000","last_undegraded":"2024-09-15T23:36:30.024511+0000","last_fullsized":"2024-09-15T23:36:30.024511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:23.983256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024984+0000","last_change":"2024-09-15T23:36:22.569943+0000","last_active":"2024-09-15T23:36:30.024984+0000","last_peered":"2024-09-15T23:36:30.024984+0000","last_clean":"2024-09-15T23:36:30.024984+0000","last_became_active":"2024-09-15T23:36:22.568617+0000","last_became_peered":"2024-09-15T23:36:22.568617+0000","last_unstale":"2024-09-15T23:36:30.024984+0000","last_undegraded":"2024-09-15T23:36:30.024984+0000","last_fullsized":"2024-09-15T23:36:30.024984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:44.229130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454281+0000","last_change":"2024-09-15T23:36:27.604270+0000","last_active":"2024-09-15T23:36:29.454281+0000","last_peered":"2024-09-15T23:36:29.454281+0000","last_clean":"2024-09-15T23:36:29.454281+0000","last_became_active":"2024-09-15T23:36:27.603416+0000","last_became_peered":"2024-09-15T23:36:27.603416+0000","last_unstale":"2024-09-15T23:36:29.454281+0000","last_undegraded":"2024-09-15T23:36:29.454281+0000","last_fullsized":"2024-09-15T23:36:29.454281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:12.747026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024835+0000","last_change":"2024-09-15T23:36:23.580585+0000","last_active":"2024-09-15T23:36:30.024835+0000","last_peered":"2024-09-15T23:36:30.024835+0000","last_clean":"2024-09-15T23:36:30.024835+0000","last_became_active":"2024-09-15T23:36:23.579003+0000","last_became_peered":"2024-09-15T23:36:23.579003+0000","last_unstale":"2024-09-15T23:36:30.024835+0000","last_undegraded":"2024-09-15T23:36:30.024835+0000","last_fullsized":"2024-09-15T23:36:30.024835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:24.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025091+0000","last_change":"2024-09-15T23:36:22.557388+0000","last_active":"2024-09-15T23:36:30.025091+0000","last_peered":"2024-09-15T23:36:30.025091+0000","last_clean":"2024-09-15T23:36:30.025091+0000","last_became_active":"2024-09-15T23:36:22.556138+0000","last_became_peered":"2024-09-15T23:36:22.556138+0000","last_unstale":"2024-09-15T23:36:30.025091+0000","last_undegraded":"2024-09-15T23:36:30.025091+0000","last_fullsized":"2024-09-15T23:36:30.025091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623557+0000","last_change":"2024-09-15T23:36:27.600449+0000","last_active":"2024-09-15T23:36:28.623557+0000","last_peered":"2024-09-15T23:36:28.623557+0000","last_clean":"2024-09-15T23:36:28.623557+0000","last_became_active":"2024-09-15T23:36:27.599937+0000","last_became_peered":"2024-09-15T23:36:27.599937+0000","last_unstale":"2024-09-15T23:36:28.623557+0000","last_undegraded":"2024-09-15T23:36:28.623557+0000","last_fullsized":"2024-09-15T23:36:28.623557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:13.541381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623785+0000","last_change":"2024-09-15T23:36:27.607667+0000","last_active":"2024-09-15T23:36:28.623785+0000","last_peered":"2024-09-15T23:36:28.623785+0000","last_clean":"2024-09-15T23:36:28.623785+0000","last_became_active":"2024-09-15T23:36:27.607227+0000","last_became_peered":"2024-09-15T23:36:27.607227+0000","last_unstale":"2024-09-15T23:36:28.623785+0000","last_undegraded":"2024-09-15T23:36:28.623785+0000","last_fullsized":"2024-09-15T23:36:28.623785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:33.572386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623619+0000","last_change":"2024-09-15T23:36:23.579014+0000","last_active":"2024-09-15T23:36:28.623619+0000","last_peered":"2024-09-15T23:36:28.623619+0000","last_clean":"2024-09-15T23:36:28.623619+0000","last_became_active":"2024-09-15T23:36:23.578709+0000","last_became_peered":"2024-09-15T23:36:23.578709+0000","last_unstale":"2024-09-15T23:36:28.623619+0000","last_undegraded":"2024-09-15T23:36:28.623619+0000","last_fullsized":"2024-09-15T23:36:28.623619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:07.005224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622050+0000","last_change":"2024-09-15T23:36:22.556649+0000","last_active":"2024-09-15T23:36:28.622050+0000","last_peered":"2024-09-15T23:36:28.622050+0000","last_clean":"2024-09-15T23:36:28.622050+0000","last_became_active":"2024-09-15T23:36:22.556332+0000","last_became_peered":"2024-09-15T23:36:22.556332+0000","last_unstale":"2024-09-15T23:36:28.622050+0000","last_undegraded":"2024-09-15T23:36:28.622050+0000","last_fullsized":"2024-09-15T23:36:28.622050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:46.855766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623003+0000","last_change":"2024-09-15T23:36:27.612447+0000","last_active":"2024-09-15T23:36:28.623003+0000","last_peered":"2024-09-15T23:36:28.623003+0000","last_clean":"2024-09-15T23:36:28.623003+0000","last_became_active":"2024-09-15T23:36:27.612007+0000","last_became_peered":"2024-09-15T23:36:27.612007+0000","last_unstale":"2024-09-15T23:36:28.623003+0000","last_undegraded":"2024-09-15T23:36:28.623003+0000","last_fullsized":"2024-09-15T23:36:28.623003+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:15.233014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023818+0000","last_change":"2024-09-15T23:36:23.592928+0000","last_active":"2024-09-15T23:36:30.023818+0000","last_peered":"2024-09-15T23:36:30.023818+0000","last_clean":"2024-09-15T23:36:30.023818+0000","last_became_active":"2024-09-15T23:36:23.592641+0000","last_became_peered":"2024-09-15T23:36:23.592641+0000","last_unstale":"2024-09-15T23:36:30.023818+0000","last_undegraded":"2024-09-15T23:36:30.023818+0000","last_fullsized":"2024-09-15T23:36:30.023818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:02.943531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622171+0000","last_change":"2024-09-15T23:36:22.569415+0000","last_active":"2024-09-15T23:36:28.622171+0000","last_peered":"2024-09-15T23:36:28.622171+0000","last_clean":"2024-09-15T23:36:28.622171+0000","last_became_active":"2024-09-15T23:36:22.569105+0000","last_became_peered":"2024-09-15T23:36:22.569105+0000","last_unstale":"2024-09-15T23:36:28.622171+0000","last_undegraded":"2024-09-15T23:36:28.622171+0000","last_fullsized":"2024-09-15T23:36:28.622171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622420+0000","last_change":"2024-09-15T23:36:22.569714+0000","last_active":"2024-09-15T23:36:28.622420+0000","last_peered":"2024-09-15T23:36:28.622420+0000","last_clean":"2024-09-15T23:36:28.622420+0000","last_became_active":"2024-09-15T23:36:22.569371+0000","last_became_peered":"2024-09-15T23:36:22.569371+0000","last_unstale":"2024-09-15T23:36:28.622420+0000","last_undegraded":"2024-09-15T23:36:28.622420+0000","last_fullsized":"2024-09-15T23:36:28.622420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:19.167117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622958+0000","last_change":"2024-09-15T23:36:23.591060+0000","last_active":"2024-09-15T23:36:28.622958+0000","last_peered":"2024-09-15T23:36:28.622958+0000","last_clean":"2024-09-15T23:36:28.622958+0000","last_became_active":"2024-09-15T23:36:23.590748+0000","last_became_peered":"2024-09-15T23:36:23.590748+0000","last_unstale":"2024-09-15T23:36:28.622958+0000","last_undegraded":"2024-09-15T23:36:28.622958+0000","last_fullsized":"2024-09-15T23:36:28.622958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:56.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624452+0000","last_change":"2024-09-15T23:36:27.602410+0000","last_active":"2024-09-15T23:36:28.624452+0000","last_peered":"2024-09-15T23:36:28.624452+0000","last_clean":"2024-09-15T23:36:28.624452+0000","last_became_active":"2024-09-15T23:36:27.602009+0000","last_became_peered":"2024-09-15T23:36:27.602009+0000","last_unstale":"2024-09-15T23:36:28.624452+0000","last_undegraded":"2024-09-15T23:36:28.624452+0000","last_fullsized":"2024-09-15T23:36:28.624452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:38.882900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024165+0000","last_change":"2024-09-15T23:36:22.562409+0000","last_active":"2024-09-15T23:36:30.024165+0000","last_peered":"2024-09-15T23:36:30.024165+0000","last_clean":"2024-09-15T23:36:30.024165+0000","last_became_active":"2024-09-15T23:36:22.561579+0000","last_became_peered":"2024-09-15T23:36:22.561579+0000","last_unstale":"2024-09-15T23:36:30.024165+0000","last_undegraded":"2024-09-15T23:36:30.024165+0000","last_fullsized":"2024-09-15T23:36:30.024165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:46.879331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023841+0000","last_change":"2024-09-15T23:36:23.584956+0000","last_active":"2024-09-15T23:36:30.023841+0000","last_peered":"2024-09-15T23:36:30.023841+0000","last_clean":"2024-09-15T23:36:30.023841+0000","last_became_active":"2024-09-15T23:36:23.584680+0000","last_became_peered":"2024-09-15T23:36:23.584680+0000","last_unstale":"2024-09-15T23:36:30.023841+0000","last_undegraded":"2024-09-15T23:36:30.023841+0000","last_fullsized":"2024-09-15T23:36:30.023841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454641+0000","last_change":"2024-09-15T23:36:27.603930+0000","last_active":"2024-09-15T23:36:29.454641+0000","last_peered":"2024-09-15T23:36:29.454641+0000","last_clean":"2024-09-15T23:36:29.454641+0000","last_became_active":"2024-09-15T23:36:27.603486+0000","last_became_peered":"2024-09-15T23:36:27.603486+0000","last_unstale":"2024-09-15T23:36:29.454641+0000","last_undegraded":"2024-09-15T23:36:29.454641+0000","last_fullsized":"2024-09-15T23:36:29.454641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:41.146991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024615+0000","last_change":"2024-09-15T23:36:23.590531+0000","last_active":"2024-09-15T23:36:30.024615+0000","last_peered":"2024-09-15T23:36:30.024615+0000","last_clean":"2024-09-15T23:36:30.024615+0000","last_became_active":"2024-09-15T23:36:23.590278+0000","last_became_peered":"2024-09-15T23:36:23.590278+0000","last_unstale":"2024-09-15T23:36:30.024615+0000","last_undegraded":"2024-09-15T23:36:30.024615+0000","last_fullsized":"2024-09-15T23:36:30.024615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454276+0000","last_change":"2024-09-15T23:36:23.580382+0000","last_active":"2024-09-15T23:36:29.454276+0000","last_peered":"2024-09-15T23:36:29.454276+0000","last_clean":"2024-09-15T23:36:29.454276+0000","last_became_active":"2024-09-15T23:36:23.580096+0000","last_became_peered":"2024-09-15T23:36:23.580096+0000","last_unstale":"2024-09-15T23:36:29.454276+0000","last_undegraded":"2024-09-15T23:36:29.454276+0000","last_fullsized":"2024-09-15T23:36:29.454276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622200+0000","last_change":"2024-09-15T23:36:23.587488+0000","last_active":"2024-09-15T23:36:28.622200+0000","last_peered":"2024-09-15T23:36:28.622200+0000","last_clean":"2024-09-15T23:36:28.622200+0000","last_became_active":"2024-09-15T23:36:23.587199+0000","last_became_peered":"2024-09-15T23:36:23.587199+0000","last_unstale":"2024-09-15T23:36:28.622200+0000","last_undegraded":"2024-09-15T23:36:28.622200+0000","last_fullsized":"2024-09-15T23:36:28.622200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:41.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023999+0000","last_change":"2024-09-15T23:36:23.583971+0000","last_active":"2024-09-15T23:36:30.023999+0000","last_peered":"2024-09-15T23:36:30.023999+0000","last_clean":"2024-09-15T23:36:30.023999+0000","last_became_active":"2024-09-15T23:36:23.583670+0000","last_became_peered":"2024-09-15T23:36:23.583670+0000","last_unstale":"2024-09-15T23:36:30.023999+0000","last_undegraded":"2024-09-15T23:36:30.023999+0000","last_fullsized":"2024-09-15T23:36:30.023999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:56.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622689+0000","last_change":"2024-09-15T23:36:23.598604+0000","last_active":"2024-09-15T23:36:28.622689+0000","last_peered":"2024-09-15T23:36:28.622689+0000","last_clean":"2024-09-15T23:36:28.622689+0000","last_became_active":"2024-09-15T23:36:23.598283+0000","last_became_peered":"2024-09-15T23:36:23.598283+0000","last_unstale":"2024-09-15T23:36:28.622689+0000","last_undegraded":"2024-09-15T23:36:28.622689+0000","last_fullsized":"2024-09-15T23:36:28.622689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:46.930120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622563+0000","last_change":"2024-09-15T23:36:23.588957+0000","last_active":"2024-09-15T23:36:28.622563+0000","last_peered":"2024-09-15T23:36:28.622563+0000","last_clean":"2024-09-15T23:36:28.622563+0000","last_became_active":"2024-09-15T23:36:23.588651+0000","last_became_peered":"2024-09-15T23:36:23.588651+0000","last_unstale":"2024-09-15T23:36:28.622563+0000","last_undegraded":"2024-09-15T23:36:28.622563+0000","last_fullsized":"2024-09-15T23:36:28.622563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:05.334429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024765+0000","last_change":"2024-09-15T23:36:23.579681+0000","last_active":"2024-09-15T23:36:30.024765+0000","last_peered":"2024-09-15T23:36:30.024765+0000","last_clean":"2024-09-15T23:36:30.024765+0000","last_became_active":"2024-09-15T23:36:23.579299+0000","last_became_peered":"2024-09-15T23:36:23.579299+0000","last_unstale":"2024-09-15T23:36:30.024765+0000","last_undegraded":"2024-09-15T23:36:30.024765+0000","last_fullsized":"2024-09-15T23:36:30.024765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:19.006869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024346+0000","last_change":"2024-09-15T23:36:23.587478+0000","last_active":"2024-09-15T23:36:30.024346+0000","last_peered":"2024-09-15T23:36:30.024346+0000","last_clean":"2024-09-15T23:36:30.024346+0000","last_became_active":"2024-09-15T23:36:23.586993+0000","last_became_peered":"2024-09-15T23:36:23.586993+0000","last_unstale":"2024-09-15T23:36:30.024346+0000","last_undegraded":"2024-09-15T23:36:30.024346+0000","last_fullsized":"2024-09-15T23:36:30.024346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:11:50.246113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622283+0000","last_change":"2024-09-15T23:36:23.594827+0000","last_active":"2024-09-15T23:36:28.622283+0000","last_peered":"2024-09-15T23:36:28.622283+0000","last_clean":"2024-09-15T23:36:28.622283+0000","last_became_active":"2024-09-15T23:36:23.594542+0000","last_became_peered":"2024-09-15T23:36:23.594542+0000","last_unstale":"2024-09-15T23:36:28.622283+0000","last_undegraded":"2024-09-15T23:36:28.622283+0000","last_fullsized":"2024-09-15T23:36:28.622283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.872038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023772+0000","last_change":"2024-09-15T23:36:23.588492+0000","last_active":"2024-09-15T23:36:30.023772+0000","last_peered":"2024-09-15T23:36:30.023772+0000","last_clean":"2024-09-15T23:36:30.023772+0000","last_became_active":"2024-09-15T23:36:23.588154+0000","last_became_peered":"2024-09-15T23:36:23.588154+0000","last_unstale":"2024-09-15T23:36:30.023772+0000","last_undegraded":"2024-09-15T23:36:30.023772+0000","last_fullsized":"2024-09-15T23:36:30.023772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:50:24.116022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023752+0000","last_change":"2024-09-15T23:36:23.584149+0000","last_active":"2024-09-15T23:36:30.023752+0000","last_peered":"2024-09-15T23:36:30.023752+0000","last_clean":"2024-09-15T23:36:30.023752+0000","last_became_active":"2024-09-15T23:36:23.583864+0000","last_became_peered":"2024-09-15T23:36:23.583864+0000","last_unstale":"2024-09-15T23:36:30.023752+0000","last_undegraded":"2024-09-15T23:36:30.023752+0000","last_fullsized":"2024-09-15T23:36:30.023752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:09.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622823+0000","last_change":"2024-09-15T23:36:23.581142+0000","last_active":"2024-09-15T23:36:28.622823+0000","last_peered":"2024-09-15T23:36:28.622823+0000","last_clean":"2024-09-15T23:36:28.622823+0000","last_became_active":"2024-09-15T23:36:23.580551+0000","last_became_peered":"2024-09-15T23:36:23.580551+0000","last_unstale":"2024-09-15T23:36:28.622823+0000","last_undegraded":"2024-09-15T23:36:28.622823+0000","last_fullsized":"2024-09-15T23:36:28.622823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:44.358074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025231+0000","last_change":"2024-09-15T23:36:23.596339+0000","last_active":"2024-09-15T23:36:30.025231+0000","last_peered":"2024-09-15T23:36:30.025231+0000","last_clean":"2024-09-15T23:36:30.025231+0000","last_became_active":"2024-09-15T23:36:23.590862+0000","last_became_peered":"2024-09-15T23:36:23.590862+0000","last_unstale":"2024-09-15T23:36:30.025231+0000","last_undegraded":"2024-09-15T23:36:30.025231+0000","last_fullsized":"2024-09-15T23:36:30.025231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:06.543007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623049+0000","last_change":"2024-09-15T23:36:23.589509+0000","last_active":"2024-09-15T23:36:28.623049+0000","last_peered":"2024-09-15T23:36:28.623049+0000","last_clean":"2024-09-15T23:36:28.623049+0000","last_became_active":"2024-09-15T23:36:23.589201+0000","last_became_peered":"2024-09-15T23:36:23.589201+0000","last_unstale":"2024-09-15T23:36:28.623049+0000","last_undegraded":"2024-09-15T23:36:28.623049+0000","last_fullsized":"2024-09-15T23:36:28.623049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:02.514262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454968+0000","last_change":"2024-09-15T23:36:23.582505+0000","last_active":"2024-09-15T23:36:29.454968+0000","last_peered":"2024-09-15T23:36:29.454968+0000","last_clean":"2024-09-15T23:36:29.454968+0000","last_became_active":"2024-09-15T23:36:23.582210+0000","last_became_peered":"2024-09-15T23:36:23.582210+0000","last_unstale":"2024-09-15T23:36:29.454968+0000","last_undegraded":"2024-09-15T23:36:29.454968+0000","last_fullsized":"2024-09-15T23:36:29.454968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623156+0000","last_change":"2024-09-15T23:36:23.590697+0000","last_active":"2024-09-15T23:36:28.623156+0000","last_peered":"2024-09-15T23:36:28.623156+0000","last_clean":"2024-09-15T23:36:28.623156+0000","last_became_active":"2024-09-15T23:36:23.590399+0000","last_became_peered":"2024-09-15T23:36:23.590399+0000","last_unstale":"2024-09-15T23:36:28.623156+0000","last_undegraded":"2024-09-15T23:36:28.623156+0000","last_fullsized":"2024-09-15T23:36:28.623156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.804071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621897+0000","last_change":"2024-09-15T23:36:23.583813+0000","last_active":"2024-09-15T23:36:28.621897+0000","last_peered":"2024-09-15T23:36:28.621897+0000","last_clean":"2024-09-15T23:36:28.621897+0000","last_became_active":"2024-09-15T23:36:23.583516+0000","last_became_peered":"2024-09-15T23:36:23.583516+0000","last_unstale":"2024-09-15T23:36:28.621897+0000","last_undegraded":"2024-09-15T23:36:28.621897+0000","last_fullsized":"2024-09-15T23:36:28.621897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:40.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023592+0000","last_change":"2024-09-15T23:36:23.587846+0000","last_active":"2024-09-15T23:36:30.023592+0000","last_peered":"2024-09-15T23:36:30.023592+0000","last_clean":"2024-09-15T23:36:30.023592+0000","last_became_active":"2024-09-15T23:36:23.587540+0000","last_became_peered":"2024-09-15T23:36:23.587540+0000","last_unstale":"2024-09-15T23:36:30.023592+0000","last_undegraded":"2024-09-15T23:36:30.023592+0000","last_fullsized":"2024-09-15T23:36:30.023592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:31.212365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621934+0000","last_change":"2024-09-15T23:36:23.589402+0000","last_active":"2024-09-15T23:36:28.621934+0000","last_peered":"2024-09-15T23:36:28.621934+0000","last_clean":"2024-09-15T23:36:28.621934+0000","last_became_active":"2024-09-15T23:36:23.589096+0000","last_became_peered":"2024-09-15T23:36:23.589096+0000","last_unstale":"2024-09-15T23:36:28.621934+0000","last_undegraded":"2024-09-15T23:36:28.621934+0000","last_fullsized":"2024-09-15T23:36:28.621934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:19.006869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621955+0000","last_change":"2024-09-15T23:36:23.598740+0000","last_active":"2024-09-15T23:36:28.621955+0000","last_peered":"2024-09-15T23:36:28.621955+0000","last_clean":"2024-09-15T23:36:28.621955+0000","last_became_active":"2024-09-15T23:36:23.598307+0000","last_became_peered":"2024-09-15T23:36:23.598307+0000","last_unstale":"2024-09-15T23:36:28.621955+0000","last_undegraded":"2024-09-15T23:36:28.621955+0000","last_fullsized":"2024-09-15T23:36:28.621955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:24.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025141+0000","last_change":"2024-09-15T23:36:23.591784+0000","last_active":"2024-09-15T23:36:30.025141+0000","last_peered":"2024-09-15T23:36:30.025141+0000","last_clean":"2024-09-15T23:36:30.025141+0000","last_became_active":"2024-09-15T23:36:23.591022+0000","last_became_peered":"2024-09-15T23:36:23.591022+0000","last_unstale":"2024-09-15T23:36:30.025141+0000","last_undegraded":"2024-09-15T23:36:30.025141+0000","last_fullsized":"2024-09-15T23:36:30.025141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:23.323481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025097+0000","last_change":"2024-09-15T23:36:23.586049+0000","last_active":"2024-09-15T23:36:30.025097+0000","last_peered":"2024-09-15T23:36:30.025097+0000","last_clean":"2024-09-15T23:36:30.025097+0000","last_became_active":"2024-09-15T23:36:23.585679+0000","last_became_peered":"2024-09-15T23:36:23.585679+0000","last_unstale":"2024-09-15T23:36:30.025097+0000","last_undegraded":"2024-09-15T23:36:30.025097+0000","last_fullsized":"2024-09-15T23:36:30.025097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:39.610334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024825+0000","last_change":"2024-09-15T23:36:23.595842+0000","last_active":"2024-09-15T23:36:30.024825+0000","last_peered":"2024-09-15T23:36:30.024825+0000","last_clean":"2024-09-15T23:36:30.024825+0000","last_became_active":"2024-09-15T23:36:23.595526+0000","last_became_peered":"2024-09-15T23:36:23.595526+0000","last_unstale":"2024-09-15T23:36:30.024825+0000","last_undegraded":"2024-09-15T23:36:30.024825+0000","last_fullsized":"2024-09-15T23:36:30.024825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:37.109784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025005+0000","last_change":"2024-09-15T23:36:23.572643+0000","last_active":"2024-09-15T23:36:30.025005+0000","last_peered":"2024-09-15T23:36:30.025005+0000","last_clean":"2024-09-15T23:36:30.025005+0000","last_became_active":"2024-09-15T23:36:23.572340+0000","last_became_peered":"2024-09-15T23:36:23.572340+0000","last_unstale":"2024-09-15T23:36:30.025005+0000","last_undegraded":"2024-09-15T23:36:30.025005+0000","last_fullsized":"2024-09-15T23:36:30.025005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023623+0000","last_change":"2024-09-15T23:36:23.592206+0000","last_active":"2024-09-15T23:36:30.023623+0000","last_peered":"2024-09-15T23:36:30.023623+0000","last_clean":"2024-09-15T23:36:30.023623+0000","last_became_active":"2024-09-15T23:36:23.590955+0000","last_became_peered":"2024-09-15T23:36:23.590955+0000","last_unstale":"2024-09-15T23:36:30.023623+0000","last_undegraded":"2024-09-15T23:36:30.023623+0000","last_fullsized":"2024-09-15T23:36:30.023623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:17.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023548+0000","last_change":"2024-09-15T23:36:23.594637+0000","last_active":"2024-09-15T23:36:30.023548+0000","last_peered":"2024-09-15T23:36:30.023548+0000","last_clean":"2024-09-15T23:36:30.023548+0000","last_became_active":"2024-09-15T23:36:23.594333+0000","last_became_peered":"2024-09-15T23:36:23.594333+0000","last_unstale":"2024-09-15T23:36:30.023548+0000","last_undegraded":"2024-09-15T23:36:30.023548+0000","last_fullsized":"2024-09-15T23:36:30.023548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:51.033435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024411+0000","last_change":"2024-09-15T23:36:23.592276+0000","last_active":"2024-09-15T23:36:30.024411+0000","last_peered":"2024-09-15T23:36:30.024411+0000","last_clean":"2024-09-15T23:36:30.024411+0000","last_became_active":"2024-09-15T23:36:23.591988+0000","last_became_peered":"2024-09-15T23:36:23.591988+0000","last_unstale":"2024-09-15T23:36:30.024411+0000","last_undegraded":"2024-09-15T23:36:30.024411+0000","last_fullsized":"2024-09-15T23:36:30.024411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024046+0000","last_change":"2024-09-15T23:36:23.596925+0000","last_active":"2024-09-15T23:36:30.024046+0000","last_peered":"2024-09-15T23:36:30.024046+0000","last_clean":"2024-09-15T23:36:30.024046+0000","last_became_active":"2024-09-15T23:36:23.596610+0000","last_became_peered":"2024-09-15T23:36:23.596610+0000","last_unstale":"2024-09-15T23:36:30.024046+0000","last_undegraded":"2024-09-15T23:36:30.024046+0000","last_fullsized":"2024-09-15T23:36:30.024046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:56.256643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023789+0000","last_change":"2024-09-15T23:36:23.591422+0000","last_active":"2024-09-15T23:36:30.023789+0000","last_peered":"2024-09-15T23:36:30.023789+0000","last_clean":"2024-09-15T23:36:30.023789+0000","last_became_active":"2024-09-15T23:36:23.590380+0000","last_became_peered":"2024-09-15T23:36:23.590380+0000","last_unstale":"2024-09-15T23:36:30.023789+0000","last_undegraded":"2024-09-15T23:36:30.023789+0000","last_fullsized":"2024-09-15T23:36:30.023789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:20.169043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024050+0000","last_change":"2024-09-15T23:36:23.592150+0000","last_active":"2024-09-15T23:36:30.024050+0000","last_peered":"2024-09-15T23:36:30.024050+0000","last_clean":"2024-09-15T23:36:30.024050+0000","last_became_active":"2024-09-15T23:36:23.591205+0000","last_became_peered":"2024-09-15T23:36:23.591205+0000","last_unstale":"2024-09-15T23:36:30.024050+0000","last_undegraded":"2024-09-15T23:36:30.024050+0000","last_fullsized":"2024-09-15T23:36:30.024050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:50.950584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024247+0000","last_change":"2024-09-15T23:36:23.593241+0000","last_active":"2024-09-15T23:36:30.024247+0000","last_peered":"2024-09-15T23:36:30.024247+0000","last_clean":"2024-09-15T23:36:30.024247+0000","last_became_active":"2024-09-15T23:36:23.591733+0000","last_became_peered":"2024-09-15T23:36:23.591733+0000","last_unstale":"2024-09-15T23:36:30.024247+0000","last_undegraded":"2024-09-15T23:36:30.024247+0000","last_fullsized":"2024-09-15T23:36:30.024247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:23.692197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024357+0000","last_change":"2024-09-15T23:36:23.596045+0000","last_active":"2024-09-15T23:36:30.024357+0000","last_peered":"2024-09-15T23:36:30.024357+0000","last_clean":"2024-09-15T23:36:30.024357+0000","last_became_active":"2024-09-15T23:36:23.595078+0000","last_became_peered":"2024-09-15T23:36:23.595078+0000","last_unstale":"2024-09-15T23:36:30.024357+0000","last_undegraded":"2024-09-15T23:36:30.024357+0000","last_fullsized":"2024-09-15T23:36:30.024357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:40.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":262144,"data_stored":103393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":270336,"data_stored":106655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":856064,"data_stored":693249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":102867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":102935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":258048,"data_stored":102811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":860160,"data_stored":693271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":107327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:36:46.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:36:46.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.756+0000 7f3521f52640 1 Processor -- start 2024-09-15T23:36:46.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.756+0000 7f3521f52640 1 -- start start 2024-09-15T23:36:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.756+0000 7f3521f52640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f351c0d0f90 0x7f351c0d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.756+0000 7f3521f52640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f351c0d0a30 con 0x7f351c0d52e0 2024-09-15T23:36:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.756+0000 7f3521f52640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f351c06dd80 con 0x7f351c15e840 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.756+0000 7f3521f52640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f351c0cfe90 con 0x7f351c0d0f90 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351affd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f351c0d52e0 0x7f351c06d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44746/0 (socket says 172.21.15.7:44746) 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351affd640 1 -- 172.21.15.7:0/1356671215 learned_addr learned my addr 172.21.15.7:0/1356671215 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351bfff640 1 --2- 172.21.15.7:0/1356671215 >> v2:172.21.15.18:3300/0 conn(0x7f351c0d0f90 0x7f351c0d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1597604332 0 0) 0x7f351c0d0a30 con 0x7f351c0d52e0 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3504003520 con 0x7f351c0d52e0 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2026434410 0 0) 0x7f3504003520 con 0x7f351c0d52e0 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 >> v2:172.21.15.18:3300/0 conn(0x7f351c0d0f90 msgr2=0x7f351c0d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 --2- 172.21.15.7:0/1356671215 >> v2:172.21.15.18:3300/0 conn(0x7f351c0d0f90 0x7f351c0d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 >> v1:172.21.15.18:6789/0 conn(0x7f351c15e840 legacy=0x7f351c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c0cf560 con 0x7f351c0d52e0 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f35100017f0 con 0x7f351c0d52e0 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 >> v1:172.21.15.7:6789/0 conn(0x7f351c0d52e0 legacy=0x7f351c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f35040039b0 0x7f3504003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35040043d0 0x7f35040048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35040050a0 0x7f350400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f351c0cfe90 con 0x7f35040043d0 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f350400bfe0 con 0x7f35040050a0 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f350400c1e0 con 0x7f35040039b0 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351affd640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35040043d0 0x7f35040048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351b7fe640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f35040039b0 0x7f3504003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.757+0000 7f351bfff640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35040050a0 0x7f350400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.758+0000 7f351affd640 1 -- 172.21.15.7:0/1356671215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f35040039b0 msgr2=0x7f3504003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.758+0000 7f351affd640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f35040039b0 0x7f3504003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.758+0000 7f351affd640 1 -- 172.21.15.7:0/1356671215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35040050a0 msgr2=0x7f350400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.758+0000 7f351affd640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35040050a0 0x7f350400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.758+0000 7f351bfff640 1 -- 172.21.15.7:0/1356671215 reap_dead start 2024-09-15T23:36:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.758+0000 7f351affd640 1 -- 172.21.15.7:0/1356671215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3504003520 con 0x7f35040043d0 2024-09-15T23:36:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.758+0000 7f351affd640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35040043d0 0x7f35040048a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f351000c160 tx=0x7f3504005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d416489eba117 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510007ca0 con 0x7f35040043d0 2024-09-15T23:36:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3510007270 con 0x7f35040043d0 2024-09-15T23:36:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f351a7fc640 1 -- 172.21.15.7:0/1356671215 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35100076a0 con 0x7f35040043d0 2024-09-15T23:36:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f3521f52640 1 -- 172.21.15.7:0/1356671215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35040043d0 msgr2=0x7f35040048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f3521f52640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35040043d0 0x7f35040048a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f351000c160 tx=0x7f3504005830 comp rx=0 tx=0).stop 2024-09-15T23:36:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f3521f52640 1 -- 172.21.15.7:0/1356671215 shutdown_connections 2024-09-15T23:36:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f3521f52640 1 --2- 172.21.15.7:0/1356671215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35040043d0 0x7f35040048a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f3521f52640 1 -- 172.21.15.7:0/1356671215 >> 172.21.15.7:0/1356671215 conn(0x7f351c08bfd0 msgr2=0x7f351c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f3521f52640 1 -- 172.21.15.7:0/1356671215 shutdown_connections 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.759+0000 7f3521f52640 1 -- 172.21.15.7:0/1356671215 wait complete. 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.760+0000 7f3521f52640 1 Processor -- start 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f3521f52640 1 -- start start 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f3521f52640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35040043d0 0x7f351c14dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f3521f52640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f351c0b8120 0x7f351c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f3521f52640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f351c150560 0x7f351c1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f3521f52640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f351c06e030 con 0x7f351c0b8120 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f3521f52640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f351c0cfb20 con 0x7f35040043d0 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f3521f52640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f351c15ec20 con 0x7f351c150560 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f351affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f351c0b8120 0x7f351c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f351affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f351c0b8120 0x7f351c0b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48476/0 (socket says 172.21.15.7:48476) 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f351affd640 1 -- 172.21.15.7:0/1599393394 learned_addr learned my addr 172.21.15.7:0/1599393394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.761+0000 7f351b7fe640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35040043d0 0x7f351c14dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f351bfff640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f351c150560 0x7f351c1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f351affd640 1 -- 172.21.15.7:0/1599393394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f351c150560 msgr2=0x7f351c1509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f351affd640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f351c150560 0x7f351c1509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f351affd640 1 -- 172.21.15.7:0/1599393394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35040043d0 msgr2=0x7f351c14dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f351affd640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35040043d0 0x7f351c14dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f351affd640 1 -- 172.21.15.7:0/1599393394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c1512d0 con 0x7f351c0b8120 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f351affd640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f351c0b8120 0x7f351c0b85a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f351000c280 tx=0x7f351c0cf730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f3503fff640 1 -- 172.21.15.7:0/1599393394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510007630 con 0x7f351c0b8120 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f3503fff640 1 -- 172.21.15.7:0/1599393394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3510042070 con 0x7f351c0b8120 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c14e720 con 0x7f351c0b8120 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f3503fff640 1 -- 172.21.15.7:0/1599393394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35100059e0 con 0x7f351c0b8120 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.762+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c14ece0 con 0x7f351c0b8120 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.763+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351c0b8ae0 con 0x7f351c0b8120 2024-09-15T23:36:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.764+0000 7f3503fff640 1 -- 172.21.15.7:0/1599393394 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3510032430 con 0x7f351c0b8120 2024-09-15T23:36:46.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.764+0000 7f3503fff640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f350806c2e0 0x7f350806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:46.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.764+0000 7f3503fff640 1 -- 172.21.15.7:0/1599393394 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f35100ab440 con 0x7f351c0b8120 2024-09-15T23:36:46.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.764+0000 7f351b7fe640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f350806c2e0 0x7f350806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:46.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.765+0000 7f351b7fe640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f350806c2e0 0x7f350806e7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f351c06b070 tx=0x7f350c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:46.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.769+0000 7f3503fff640 1 -- 172.21.15.7:0/1599393394 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f351004b020 con 0x7f351c0b8120 2024-09-15T23:36:46.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.940+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f351c0d13c0 con 0x7f350806c2e0 2024-09-15T23:36:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.948+0000 7f3503fff640 1 -- 172.21.15.7:0/1599393394 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7f351c0d13c0 con 0x7f350806c2e0 2024-09-15T23:36:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:46.955 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:36:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f350806c2e0 msgr2=0x7f350806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f350806c2e0 0x7f350806e7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f351c06b070 tx=0x7f350c009040 comp rx=0 tx=0).stop 2024-09-15T23:36:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f351c0b8120 msgr2=0x7f351c0b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f351c0b8120 0x7f351c0b85a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f351000c280 tx=0x7f351c0cf730 comp rx=0 tx=0).stop 2024-09-15T23:36:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 shutdown_connections 2024-09-15T23:36:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f350806c2e0 0x7f350806e7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f351c150560 0x7f351c1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f351c0b8120 0x7f351c0b85a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 --2- 172.21.15.7:0/1599393394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35040043d0 0x7f351c14dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:46.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 >> 172.21.15.7:0/1599393394 conn(0x7f351c08bfd0 msgr2=0x7f351c15aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:46.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 shutdown_connections 2024-09-15T23:36:46.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:46.952+0000 7f3521f52640 1 -- 172.21.15.7:0/1599393394 wait complete. 2024-09-15T23:36:46.964 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-15T23:36:45.816409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252752,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721968,"statfs":{"total":773094113280,"available":772835295232,"internally_reserved":0,"allocated":3317760,"data_stored":2012508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003016"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623447+0000","last_change":"2024-09-15T23:36:22.558162+0000","last_active":"2024-09-15T23:36:28.623447+0000","last_peered":"2024-09-15T23:36:28.623447+0000","last_clean":"2024-09-15T23:36:28.623447+0000","last_became_active":"2024-09-15T23:36:22.557752+0000","last_became_peered":"2024-09-15T23:36:22.557752+0000","last_unstale":"2024-09-15T23:36:28.623447+0000","last_undegraded":"2024-09-15T23:36:28.623447+0000","last_fullsized":"2024-09-15T23:36:28.623447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:17.806236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024436+0000","last_change":"2024-09-15T23:36:23.591313+0000","last_active":"2024-09-15T23:36:30.024436+0000","last_peered":"2024-09-15T23:36:30.024436+0000","last_clean":"2024-09-15T23:36:30.024436+0000","last_became_active":"2024-09-15T23:36:23.590422+0000","last_became_peered":"2024-09-15T23:36:23.590422+0000","last_unstale":"2024-09-15T23:36:30.024436+0000","last_undegraded":"2024-09-15T23:36:30.024436+0000","last_fullsized":"2024-09-15T23:36:30.024436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454338+0000","last_change":"2024-09-15T23:36:27.609388+0000","last_active":"2024-09-15T23:36:29.454338+0000","last_peered":"2024-09-15T23:36:29.454338+0000","last_clean":"2024-09-15T23:36:29.454338+0000","last_became_active":"2024-09-15T23:36:27.608999+0000","last_became_peered":"2024-09-15T23:36:27.608999+0000","last_unstale":"2024-09-15T23:36:29.454338+0000","last_undegraded":"2024-09-15T23:36:29.454338+0000","last_fullsized":"2024-09-15T23:36:29.454338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:30.088730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024564+0000","last_change":"2024-09-15T23:36:22.562097+0000","last_active":"2024-09-15T23:36:30.024564+0000","last_peered":"2024-09-15T23:36:30.024564+0000","last_clean":"2024-09-15T23:36:30.024564+0000","last_became_active":"2024-09-15T23:36:22.561356+0000","last_became_peered":"2024-09-15T23:36:22.561356+0000","last_unstale":"2024-09-15T23:36:30.024564+0000","last_undegraded":"2024-09-15T23:36:30.024564+0000","last_fullsized":"2024-09-15T23:36:30.024564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:46.482072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024461+0000","last_change":"2024-09-15T23:36:23.589236+0000","last_active":"2024-09-15T23:36:30.024461+0000","last_peered":"2024-09-15T23:36:30.024461+0000","last_clean":"2024-09-15T23:36:30.024461+0000","last_became_active":"2024-09-15T23:36:23.588873+0000","last_became_peered":"2024-09-15T23:36:23.588873+0000","last_unstale":"2024-09-15T23:36:30.024461+0000","last_undegraded":"2024-09-15T23:36:30.024461+0000","last_fullsized":"2024-09-15T23:36:30.024461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:18.209937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024371+0000","last_change":"2024-09-15T23:36:27.611526+0000","last_active":"2024-09-15T23:36:30.024371+0000","last_peered":"2024-09-15T23:36:30.024371+0000","last_clean":"2024-09-15T23:36:30.024371+0000","last_became_active":"2024-09-15T23:36:27.610759+0000","last_became_peered":"2024-09-15T23:36:27.610759+0000","last_unstale":"2024-09-15T23:36:30.024371+0000","last_undegraded":"2024-09-15T23:36:30.024371+0000","last_fullsized":"2024-09-15T23:36:30.024371+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:06.783968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025217+0000","last_change":"2024-09-15T23:36:22.568198+0000","last_active":"2024-09-15T23:36:30.025217+0000","last_peered":"2024-09-15T23:36:30.025217+0000","last_clean":"2024-09-15T23:36:30.025217+0000","last_became_active":"2024-09-15T23:36:22.567888+0000","last_became_peered":"2024-09-15T23:36:22.567888+0000","last_unstale":"2024-09-15T23:36:30.025217+0000","last_undegraded":"2024-09-15T23:36:30.025217+0000","last_fullsized":"2024-09-15T23:36:30.025217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:14:47.610942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024413+0000","last_change":"2024-09-15T23:36:23.593279+0000","last_active":"2024-09-15T23:36:30.024413+0000","last_peered":"2024-09-15T23:36:30.024413+0000","last_clean":"2024-09-15T23:36:30.024413+0000","last_became_active":"2024-09-15T23:36:23.592921+0000","last_became_peered":"2024-09-15T23:36:23.592921+0000","last_unstale":"2024-09-15T23:36:30.024413+0000","last_undegraded":"2024-09-15T23:36:30.024413+0000","last_fullsized":"2024-09-15T23:36:30.024413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:40.194400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024320+0000","last_change":"2024-09-15T23:36:27.612903+0000","last_active":"2024-09-15T23:36:30.024320+0000","last_peered":"2024-09-15T23:36:30.024320+0000","last_clean":"2024-09-15T23:36:30.024320+0000","last_became_active":"2024-09-15T23:36:27.612441+0000","last_became_peered":"2024-09-15T23:36:27.612441+0000","last_unstale":"2024-09-15T23:36:30.024320+0000","last_undegraded":"2024-09-15T23:36:30.024320+0000","last_fullsized":"2024-09-15T23:36:30.024320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:43.647541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025323+0000","last_change":"2024-09-15T23:36:22.566488+0000","last_active":"2024-09-15T23:36:30.025323+0000","last_peered":"2024-09-15T23:36:30.025323+0000","last_clean":"2024-09-15T23:36:30.025323+0000","last_became_active":"2024-09-15T23:36:22.566016+0000","last_became_peered":"2024-09-15T23:36:22.566016+0000","last_unstale":"2024-09-15T23:36:30.025323+0000","last_undegraded":"2024-09-15T23:36:30.025323+0000","last_fullsized":"2024-09-15T23:36:30.025323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:45.276407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622706+0000","last_change":"2024-09-15T23:36:23.590254+0000","last_active":"2024-09-15T23:36:28.622706+0000","last_peered":"2024-09-15T23:36:28.622706+0000","last_clean":"2024-09-15T23:36:28.622706+0000","last_became_active":"2024-09-15T23:36:23.589902+0000","last_became_peered":"2024-09-15T23:36:23.589902+0000","last_unstale":"2024-09-15T23:36:28.622706+0000","last_undegraded":"2024-09-15T23:36:28.622706+0000","last_fullsized":"2024-09-15T23:36:28.622706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:37.196308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024145+0000","last_change":"2024-09-15T23:36:27.603610+0000","last_active":"2024-09-15T23:36:30.024145+0000","last_peered":"2024-09-15T23:36:30.024145+0000","last_clean":"2024-09-15T23:36:30.024145+0000","last_became_active":"2024-09-15T23:36:27.603177+0000","last_became_peered":"2024-09-15T23:36:27.603177+0000","last_unstale":"2024-09-15T23:36:30.024145+0000","last_undegraded":"2024-09-15T23:36:30.024145+0000","last_fullsized":"2024-09-15T23:36:30.024145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:55.070642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622285+0000","last_change":"2024-09-15T23:36:23.576248+0000","last_active":"2024-09-15T23:36:28.622285+0000","last_peered":"2024-09-15T23:36:28.622285+0000","last_clean":"2024-09-15T23:36:28.622285+0000","last_became_active":"2024-09-15T23:36:23.575715+0000","last_became_peered":"2024-09-15T23:36:23.575715+0000","last_unstale":"2024-09-15T23:36:28.622285+0000","last_undegraded":"2024-09-15T23:36:28.622285+0000","last_fullsized":"2024-09-15T23:36:28.622285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:59.081263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622550+0000","last_change":"2024-09-15T23:36:22.555862+0000","last_active":"2024-09-15T23:36:28.622550+0000","last_peered":"2024-09-15T23:36:28.622550+0000","last_clean":"2024-09-15T23:36:28.622550+0000","last_became_active":"2024-09-15T23:36:22.555589+0000","last_became_peered":"2024-09-15T23:36:22.555589+0000","last_unstale":"2024-09-15T23:36:28.622550+0000","last_undegraded":"2024-09-15T23:36:28.622550+0000","last_fullsized":"2024-09-15T23:36:28.622550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:44.229130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622578+0000","last_change":"2024-09-15T23:36:27.602404+0000","last_active":"2024-09-15T23:36:28.622578+0000","last_peered":"2024-09-15T23:36:28.622578+0000","last_clean":"2024-09-15T23:36:28.622578+0000","last_became_active":"2024-09-15T23:36:27.601991+0000","last_became_peered":"2024-09-15T23:36:27.601991+0000","last_unstale":"2024-09-15T23:36:28.622578+0000","last_undegraded":"2024-09-15T23:36:28.622578+0000","last_fullsized":"2024-09-15T23:36:28.622578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:00.662926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454749+0000","last_change":"2024-09-15T23:36:23.591252+0000","last_active":"2024-09-15T23:36:29.454749+0000","last_peered":"2024-09-15T23:36:29.454749+0000","last_clean":"2024-09-15T23:36:29.454749+0000","last_became_active":"2024-09-15T23:36:23.590976+0000","last_became_peered":"2024-09-15T23:36:23.590976+0000","last_unstale":"2024-09-15T23:36:29.454749+0000","last_undegraded":"2024-09-15T23:36:29.454749+0000","last_fullsized":"2024-09-15T23:36:29.454749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622161+0000","last_change":"2024-09-15T23:36:22.560352+0000","last_active":"2024-09-15T23:36:28.622161+0000","last_peered":"2024-09-15T23:36:28.622161+0000","last_clean":"2024-09-15T23:36:28.622161+0000","last_became_active":"2024-09-15T23:36:22.560052+0000","last_became_peered":"2024-09-15T23:36:22.560052+0000","last_unstale":"2024-09-15T23:36:28.622161+0000","last_undegraded":"2024-09-15T23:36:28.622161+0000","last_fullsized":"2024-09-15T23:36:28.622161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023452+0000","last_change":"2024-09-15T23:36:27.598358+0000","last_active":"2024-09-15T23:36:30.023452+0000","last_peered":"2024-09-15T23:36:30.023452+0000","last_clean":"2024-09-15T23:36:30.023452+0000","last_became_active":"2024-09-15T23:36:27.597941+0000","last_became_peered":"2024-09-15T23:36:27.597941+0000","last_unstale":"2024-09-15T23:36:30.023452+0000","last_undegraded":"2024-09-15T23:36:30.023452+0000","last_fullsized":"2024-09-15T23:36:30.023452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:54.115527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622271+0000","last_change":"2024-09-15T23:36:23.581091+0000","last_active":"2024-09-15T23:36:28.622271+0000","last_peered":"2024-09-15T23:36:28.622271+0000","last_clean":"2024-09-15T23:36:28.622271+0000","last_became_active":"2024-09-15T23:36:23.580807+0000","last_became_peered":"2024-09-15T23:36:23.580807+0000","last_unstale":"2024-09-15T23:36:28.622271+0000","last_undegraded":"2024-09-15T23:36:28.622271+0000","last_fullsized":"2024-09-15T23:36:28.622271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:01.465502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623733+0000","last_change":"2024-09-15T23:36:22.569473+0000","last_active":"2024-09-15T23:36:28.623733+0000","last_peered":"2024-09-15T23:36:28.623733+0000","last_clean":"2024-09-15T23:36:28.623733+0000","last_became_active":"2024-09-15T23:36:22.569168+0000","last_became_peered":"2024-09-15T23:36:22.569168+0000","last_unstale":"2024-09-15T23:36:28.623733+0000","last_undegraded":"2024-09-15T23:36:28.623733+0000","last_fullsized":"2024-09-15T23:36:28.623733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:46.065174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454933+0000","last_change":"2024-09-15T23:36:27.605801+0000","last_active":"2024-09-15T23:36:29.454933+0000","last_peered":"2024-09-15T23:36:29.454933+0000","last_clean":"2024-09-15T23:36:29.454933+0000","last_became_active":"2024-09-15T23:36:27.605416+0000","last_became_peered":"2024-09-15T23:36:27.605416+0000","last_unstale":"2024-09-15T23:36:29.454933+0000","last_undegraded":"2024-09-15T23:36:29.454933+0000","last_fullsized":"2024-09-15T23:36:29.454933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:27.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622056+0000","last_change":"2024-09-15T23:36:23.587550+0000","last_active":"2024-09-15T23:36:28.622056+0000","last_peered":"2024-09-15T23:36:28.622056+0000","last_clean":"2024-09-15T23:36:28.622056+0000","last_became_active":"2024-09-15T23:36:23.587213+0000","last_became_peered":"2024-09-15T23:36:23.587213+0000","last_unstale":"2024-09-15T23:36:28.622056+0000","last_undegraded":"2024-09-15T23:36:28.622056+0000","last_fullsized":"2024-09-15T23:36:28.622056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:15.353430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023245+0000","last_change":"2024-09-15T23:36:22.569502+0000","last_active":"2024-09-15T23:36:30.023245+0000","last_peered":"2024-09-15T23:36:30.023245+0000","last_clean":"2024-09-15T23:36:30.023245+0000","last_became_active":"2024-09-15T23:36:22.567715+0000","last_became_peered":"2024-09-15T23:36:22.567715+0000","last_unstale":"2024-09-15T23:36:30.023245+0000","last_undegraded":"2024-09-15T23:36:30.023245+0000","last_fullsized":"2024-09-15T23:36:30.023245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024741+0000","last_change":"2024-09-15T23:36:27.612217+0000","last_active":"2024-09-15T23:36:30.024741+0000","last_peered":"2024-09-15T23:36:30.024741+0000","last_clean":"2024-09-15T23:36:30.024741+0000","last_became_active":"2024-09-15T23:36:27.611783+0000","last_became_peered":"2024-09-15T23:36:27.611783+0000","last_unstale":"2024-09-15T23:36:30.024741+0000","last_undegraded":"2024-09-15T23:36:30.024741+0000","last_fullsized":"2024-09-15T23:36:30.024741+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:45.020968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623852+0000","last_change":"2024-09-15T23:36:23.576178+0000","last_active":"2024-09-15T23:36:28.623852+0000","last_peered":"2024-09-15T23:36:28.623852+0000","last_clean":"2024-09-15T23:36:28.623852+0000","last_became_active":"2024-09-15T23:36:23.575823+0000","last_became_peered":"2024-09-15T23:36:23.575823+0000","last_unstale":"2024-09-15T23:36:28.623852+0000","last_undegraded":"2024-09-15T23:36:28.623852+0000","last_fullsized":"2024-09-15T23:36:28.623852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:17.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623244+0000","last_change":"2024-09-15T23:36:22.570936+0000","last_active":"2024-09-15T23:36:28.623244+0000","last_peered":"2024-09-15T23:36:28.623244+0000","last_clean":"2024-09-15T23:36:28.623244+0000","last_became_active":"2024-09-15T23:36:22.569461+0000","last_became_peered":"2024-09-15T23:36:22.569461+0000","last_unstale":"2024-09-15T23:36:28.623244+0000","last_undegraded":"2024-09-15T23:36:28.623244+0000","last_fullsized":"2024-09-15T23:36:28.623244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:38.263694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024707+0000","last_change":"2024-09-15T23:36:27.605878+0000","last_active":"2024-09-15T23:36:30.024707+0000","last_peered":"2024-09-15T23:36:30.024707+0000","last_clean":"2024-09-15T23:36:30.024707+0000","last_became_active":"2024-09-15T23:36:27.605448+0000","last_became_peered":"2024-09-15T23:36:27.605448+0000","last_unstale":"2024-09-15T23:36:30.024707+0000","last_undegraded":"2024-09-15T23:36:30.024707+0000","last_fullsized":"2024-09-15T23:36:30.024707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:00.293850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623345+0000","last_change":"2024-09-15T23:36:23.590802+0000","last_active":"2024-09-15T23:36:28.623345+0000","last_peered":"2024-09-15T23:36:28.623345+0000","last_clean":"2024-09-15T23:36:28.623345+0000","last_became_active":"2024-09-15T23:36:23.590505+0000","last_became_peered":"2024-09-15T23:36:23.590505+0000","last_unstale":"2024-09-15T23:36:28.623345+0000","last_undegraded":"2024-09-15T23:36:28.623345+0000","last_fullsized":"2024-09-15T23:36:28.623345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:50.950584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624097+0000","last_change":"2024-09-15T23:36:22.569675+0000","last_active":"2024-09-15T23:36:28.624097+0000","last_peered":"2024-09-15T23:36:28.624097+0000","last_clean":"2024-09-15T23:36:28.624097+0000","last_became_active":"2024-09-15T23:36:22.569386+0000","last_became_peered":"2024-09-15T23:36:22.569386+0000","last_unstale":"2024-09-15T23:36:28.624097+0000","last_undegraded":"2024-09-15T23:36:28.624097+0000","last_fullsized":"2024-09-15T23:36:28.624097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:00.420124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622250+0000","last_change":"2024-09-15T23:36:27.601084+0000","last_active":"2024-09-15T23:36:28.622250+0000","last_peered":"2024-09-15T23:36:28.622250+0000","last_clean":"2024-09-15T23:36:28.622250+0000","last_became_active":"2024-09-15T23:36:27.600671+0000","last_became_peered":"2024-09-15T23:36:27.600671+0000","last_unstale":"2024-09-15T23:36:28.622250+0000","last_undegraded":"2024-09-15T23:36:28.622250+0000","last_fullsized":"2024-09-15T23:36:28.622250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:28.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622694+0000","last_change":"2024-09-15T23:36:23.595285+0000","last_active":"2024-09-15T23:36:28.622694+0000","last_peered":"2024-09-15T23:36:28.622694+0000","last_clean":"2024-09-15T23:36:28.622694+0000","last_became_active":"2024-09-15T23:36:23.594596+0000","last_became_peered":"2024-09-15T23:36:23.594596+0000","last_unstale":"2024-09-15T23:36:28.622694+0000","last_undegraded":"2024-09-15T23:36:28.622694+0000","last_fullsized":"2024-09-15T23:36:28.622694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:39.265619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623245+0000","last_change":"2024-09-15T23:36:22.553052+0000","last_active":"2024-09-15T23:36:28.623245+0000","last_peered":"2024-09-15T23:36:28.623245+0000","last_clean":"2024-09-15T23:36:28.623245+0000","last_became_active":"2024-09-15T23:36:22.552766+0000","last_became_peered":"2024-09-15T23:36:22.552766+0000","last_unstale":"2024-09-15T23:36:28.623245+0000","last_undegraded":"2024-09-15T23:36:28.623245+0000","last_fullsized":"2024-09-15T23:36:28.623245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:45.928194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621909+0000","last_change":"2024-09-15T23:36:27.601501+0000","last_active":"2024-09-15T23:36:28.621909+0000","last_peered":"2024-09-15T23:36:28.621909+0000","last_clean":"2024-09-15T23:36:28.621909+0000","last_became_active":"2024-09-15T23:36:27.601071+0000","last_became_peered":"2024-09-15T23:36:27.601071+0000","last_unstale":"2024-09-15T23:36:28.621909+0000","last_undegraded":"2024-09-15T23:36:28.621909+0000","last_fullsized":"2024-09-15T23:36:28.621909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:03.241943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024193+0000","last_change":"2024-09-15T23:36:23.591415+0000","last_active":"2024-09-15T23:36:30.024193+0000","last_peered":"2024-09-15T23:36:30.024193+0000","last_clean":"2024-09-15T23:36:30.024193+0000","last_became_active":"2024-09-15T23:36:23.591132+0000","last_became_peered":"2024-09-15T23:36:23.591132+0000","last_unstale":"2024-09-15T23:36:30.024193+0000","last_undegraded":"2024-09-15T23:36:30.024193+0000","last_fullsized":"2024-09-15T23:36:30.024193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024034+0000","last_change":"2024-09-15T23:36:22.561640+0000","last_active":"2024-09-15T23:36:30.024034+0000","last_peered":"2024-09-15T23:36:30.024034+0000","last_clean":"2024-09-15T23:36:30.024034+0000","last_became_active":"2024-09-15T23:36:22.561218+0000","last_became_peered":"2024-09-15T23:36:22.561218+0000","last_unstale":"2024-09-15T23:36:30.024034+0000","last_undegraded":"2024-09-15T23:36:30.024034+0000","last_fullsized":"2024-09-15T23:36:30.024034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:06:31.226510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621863+0000","last_change":"2024-09-15T23:36:27.606889+0000","last_active":"2024-09-15T23:36:28.621863+0000","last_peered":"2024-09-15T23:36:28.621863+0000","last_clean":"2024-09-15T23:36:28.621863+0000","last_became_active":"2024-09-15T23:36:27.606466+0000","last_became_peered":"2024-09-15T23:36:27.606466+0000","last_unstale":"2024-09-15T23:36:28.621863+0000","last_undegraded":"2024-09-15T23:36:28.621863+0000","last_fullsized":"2024-09-15T23:36:28.621863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:55.070642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023954+0000","last_change":"2024-09-15T23:36:23.596605+0000","last_active":"2024-09-15T23:36:30.023954+0000","last_peered":"2024-09-15T23:36:30.023954+0000","last_clean":"2024-09-15T23:36:30.023954+0000","last_became_active":"2024-09-15T23:36:23.596318+0000","last_became_peered":"2024-09-15T23:36:23.596318+0000","last_unstale":"2024-09-15T23:36:30.023954+0000","last_undegraded":"2024-09-15T23:36:30.023954+0000","last_fullsized":"2024-09-15T23:36:30.023954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:02.746761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622853+0000","last_change":"2024-09-15T23:36:22.559703+0000","last_active":"2024-09-15T23:36:28.622853+0000","last_peered":"2024-09-15T23:36:28.622853+0000","last_clean":"2024-09-15T23:36:28.622853+0000","last_became_active":"2024-09-15T23:36:22.559271+0000","last_became_peered":"2024-09-15T23:36:22.559271+0000","last_unstale":"2024-09-15T23:36:28.622853+0000","last_undegraded":"2024-09-15T23:36:28.622853+0000","last_fullsized":"2024-09-15T23:36:28.622853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:23.347088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622571+0000","last_change":"2024-09-15T23:36:27.598188+0000","last_active":"2024-09-15T23:36:28.622571+0000","last_peered":"2024-09-15T23:36:28.622571+0000","last_clean":"2024-09-15T23:36:28.622571+0000","last_became_active":"2024-09-15T23:36:27.596219+0000","last_became_peered":"2024-09-15T23:36:27.596219+0000","last_unstale":"2024-09-15T23:36:28.622571+0000","last_undegraded":"2024-09-15T23:36:28.622571+0000","last_fullsized":"2024-09-15T23:36:28.622571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:23.776974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023619+0000","last_change":"2024-09-15T23:36:23.582672+0000","last_active":"2024-09-15T23:36:30.023619+0000","last_peered":"2024-09-15T23:36:30.023619+0000","last_clean":"2024-09-15T23:36:30.023619+0000","last_became_active":"2024-09-15T23:36:23.582388+0000","last_became_peered":"2024-09-15T23:36:23.582388+0000","last_unstale":"2024-09-15T23:36:30.023619+0000","last_undegraded":"2024-09-15T23:36:30.023619+0000","last_fullsized":"2024-09-15T23:36:30.023619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:07.005224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454431+0000","last_change":"2024-09-15T23:36:22.561458+0000","last_active":"2024-09-15T23:36:29.454431+0000","last_peered":"2024-09-15T23:36:29.454431+0000","last_clean":"2024-09-15T23:36:29.454431+0000","last_became_active":"2024-09-15T23:36:22.561171+0000","last_became_peered":"2024-09-15T23:36:22.561171+0000","last_unstale":"2024-09-15T23:36:29.454431+0000","last_undegraded":"2024-09-15T23:36:29.454431+0000","last_fullsized":"2024-09-15T23:36:29.454431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:27.731211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622564+0000","last_change":"2024-09-15T23:36:27.597617+0000","last_active":"2024-09-15T23:36:28.622564+0000","last_peered":"2024-09-15T23:36:28.622564+0000","last_clean":"2024-09-15T23:36:28.622564+0000","last_became_active":"2024-09-15T23:36:27.597224+0000","last_became_peered":"2024-09-15T23:36:27.597224+0000","last_unstale":"2024-09-15T23:36:28.622564+0000","last_undegraded":"2024-09-15T23:36:28.622564+0000","last_fullsized":"2024-09-15T23:36:28.622564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:06.783968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621714+0000","last_change":"2024-09-15T23:36:23.590567+0000","last_active":"2024-09-15T23:36:28.621714+0000","last_peered":"2024-09-15T23:36:28.621714+0000","last_clean":"2024-09-15T23:36:28.621714+0000","last_became_active":"2024-09-15T23:36:23.590272+0000","last_became_peered":"2024-09-15T23:36:23.590272+0000","last_unstale":"2024-09-15T23:36:28.621714+0000","last_undegraded":"2024-09-15T23:36:28.621714+0000","last_fullsized":"2024-09-15T23:36:28.621714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622169+0000","last_change":"2024-09-15T23:36:22.560601+0000","last_active":"2024-09-15T23:36:28.622169+0000","last_peered":"2024-09-15T23:36:28.622169+0000","last_clean":"2024-09-15T23:36:28.622169+0000","last_became_active":"2024-09-15T23:36:22.560306+0000","last_became_peered":"2024-09-15T23:36:22.560306+0000","last_unstale":"2024-09-15T23:36:28.622169+0000","last_undegraded":"2024-09-15T23:36:28.622169+0000","last_fullsized":"2024-09-15T23:36:28.622169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:38.283320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024190+0000","last_change":"2024-09-15T23:36:27.610042+0000","last_active":"2024-09-15T23:36:30.024190+0000","last_peered":"2024-09-15T23:36:30.024190+0000","last_clean":"2024-09-15T23:36:30.024190+0000","last_became_active":"2024-09-15T23:36:27.609630+0000","last_became_peered":"2024-09-15T23:36:27.609630+0000","last_unstale":"2024-09-15T23:36:30.024190+0000","last_undegraded":"2024-09-15T23:36:30.024190+0000","last_fullsized":"2024-09-15T23:36:30.024190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:03.827160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624171+0000","last_change":"2024-09-15T23:36:23.580635+0000","last_active":"2024-09-15T23:36:28.624171+0000","last_peered":"2024-09-15T23:36:28.624171+0000","last_clean":"2024-09-15T23:36:28.624171+0000","last_became_active":"2024-09-15T23:36:23.580348+0000","last_became_peered":"2024-09-15T23:36:23.580348+0000","last_unstale":"2024-09-15T23:36:28.624171+0000","last_undegraded":"2024-09-15T23:36:28.624171+0000","last_fullsized":"2024-09-15T23:36:28.624171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:03.646324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622420+0000","last_change":"2024-09-15T23:36:22.552162+0000","last_active":"2024-09-15T23:36:28.622420+0000","last_peered":"2024-09-15T23:36:28.622420+0000","last_clean":"2024-09-15T23:36:28.622420+0000","last_became_active":"2024-09-15T23:36:22.551788+0000","last_became_peered":"2024-09-15T23:36:22.551788+0000","last_unstale":"2024-09-15T23:36:28.622420+0000","last_undegraded":"2024-09-15T23:36:28.622420+0000","last_fullsized":"2024-09-15T23:36:28.622420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:04.332504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023778+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:36:30.023778+0000","last_peered":"2024-09-15T23:36:30.023778+0000","last_clean":"2024-09-15T23:36:30.023778+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:36:30.023778+0000","last_undegraded":"2024-09-15T23:36:30.023778+0000","last_fullsized":"2024-09-15T23:36:30.023778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623010+0000","last_change":"2024-09-15T23:36:27.604371+0000","last_active":"2024-09-15T23:36:28.623010+0000","last_peered":"2024-09-15T23:36:28.623010+0000","last_clean":"2024-09-15T23:36:28.623010+0000","last_became_active":"2024-09-15T23:36:27.603926+0000","last_became_peered":"2024-09-15T23:36:27.603926+0000","last_unstale":"2024-09-15T23:36:28.623010+0000","last_undegraded":"2024-09-15T23:36:28.623010+0000","last_fullsized":"2024-09-15T23:36:28.623010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:41.910981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622897+0000","last_change":"2024-09-15T23:36:23.575297+0000","last_active":"2024-09-15T23:36:28.622897+0000","last_peered":"2024-09-15T23:36:28.622897+0000","last_clean":"2024-09-15T23:36:28.622897+0000","last_became_active":"2024-09-15T23:36:23.574997+0000","last_became_peered":"2024-09-15T23:36:23.574997+0000","last_unstale":"2024-09-15T23:36:28.622897+0000","last_undegraded":"2024-09-15T23:36:28.622897+0000","last_fullsized":"2024-09-15T23:36:28.622897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:24.495389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622829+0000","last_change":"2024-09-15T23:36:22.569366+0000","last_active":"2024-09-15T23:36:28.622829+0000","last_peered":"2024-09-15T23:36:28.622829+0000","last_clean":"2024-09-15T23:36:28.622829+0000","last_became_active":"2024-09-15T23:36:22.569043+0000","last_became_peered":"2024-09-15T23:36:22.569043+0000","last_unstale":"2024-09-15T23:36:28.622829+0000","last_undegraded":"2024-09-15T23:36:28.622829+0000","last_fullsized":"2024-09-15T23:36:28.622829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:55.907462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024315+0000","last_change":"2024-09-15T23:36:27.611141+0000","last_active":"2024-09-15T23:36:30.024315+0000","last_peered":"2024-09-15T23:36:30.024315+0000","last_clean":"2024-09-15T23:36:30.024315+0000","last_became_active":"2024-09-15T23:36:27.610654+0000","last_became_peered":"2024-09-15T23:36:27.610654+0000","last_unstale":"2024-09-15T23:36:30.024315+0000","last_undegraded":"2024-09-15T23:36:30.024315+0000","last_fullsized":"2024-09-15T23:36:30.024315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:03.241943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025277+0000","last_change":"2024-09-15T23:36:23.592044+0000","last_active":"2024-09-15T23:36:30.025277+0000","last_peered":"2024-09-15T23:36:30.025277+0000","last_clean":"2024-09-15T23:36:30.025277+0000","last_became_active":"2024-09-15T23:36:23.591419+0000","last_became_peered":"2024-09-15T23:36:23.591419+0000","last_unstale":"2024-09-15T23:36:30.025277+0000","last_undegraded":"2024-09-15T23:36:30.025277+0000","last_fullsized":"2024-09-15T23:36:30.025277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:59.204736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024210+0000","last_change":"2024-09-15T23:36:22.568129+0000","last_active":"2024-09-15T23:36:30.024210+0000","last_peered":"2024-09-15T23:36:30.024210+0000","last_clean":"2024-09-15T23:36:30.024210+0000","last_became_active":"2024-09-15T23:36:22.567909+0000","last_became_peered":"2024-09-15T23:36:22.567909+0000","last_unstale":"2024-09-15T23:36:30.024210+0000","last_undegraded":"2024-09-15T23:36:30.024210+0000","last_fullsized":"2024-09-15T23:36:30.024210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:46.065174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024320+0000","last_change":"2024-09-15T23:36:27.610796+0000","last_active":"2024-09-15T23:36:30.024320+0000","last_peered":"2024-09-15T23:36:30.024320+0000","last_clean":"2024-09-15T23:36:30.024320+0000","last_became_active":"2024-09-15T23:36:27.610355+0000","last_became_peered":"2024-09-15T23:36:27.610355+0000","last_unstale":"2024-09-15T23:36:30.024320+0000","last_undegraded":"2024-09-15T23:36:30.024320+0000","last_fullsized":"2024-09-15T23:36:30.024320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:22.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024449+0000","last_change":"2024-09-15T23:36:23.592411+0000","last_active":"2024-09-15T23:36:30.024449+0000","last_peered":"2024-09-15T23:36:30.024449+0000","last_clean":"2024-09-15T23:36:30.024449+0000","last_became_active":"2024-09-15T23:36:23.592130+0000","last_became_peered":"2024-09-15T23:36:23.592130+0000","last_unstale":"2024-09-15T23:36:30.024449+0000","last_undegraded":"2024-09-15T23:36:30.024449+0000","last_fullsized":"2024-09-15T23:36:30.024449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.872038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623974+0000","last_change":"2024-09-15T23:36:22.560112+0000","last_active":"2024-09-15T23:36:28.623974+0000","last_peered":"2024-09-15T23:36:28.623974+0000","last_clean":"2024-09-15T23:36:28.623974+0000","last_became_active":"2024-09-15T23:36:22.559811+0000","last_became_peered":"2024-09-15T23:36:22.559811+0000","last_unstale":"2024-09-15T23:36:28.623974+0000","last_undegraded":"2024-09-15T23:36:28.623974+0000","last_fullsized":"2024-09-15T23:36:28.623974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:46.482072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621872+0000","last_change":"2024-09-15T23:36:27.603263+0000","last_active":"2024-09-15T23:36:28.621872+0000","last_peered":"2024-09-15T23:36:28.621872+0000","last_clean":"2024-09-15T23:36:28.621872+0000","last_became_active":"2024-09-15T23:36:27.602856+0000","last_became_peered":"2024-09-15T23:36:27.602856+0000","last_unstale":"2024-09-15T23:36:28.621872+0000","last_undegraded":"2024-09-15T23:36:28.621872+0000","last_fullsized":"2024-09-15T23:36:28.621872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:38.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024861+0000","last_change":"2024-09-15T23:36:23.594876+0000","last_active":"2024-09-15T23:36:30.024861+0000","last_peered":"2024-09-15T23:36:30.024861+0000","last_clean":"2024-09-15T23:36:30.024861+0000","last_became_active":"2024-09-15T23:36:23.594592+0000","last_became_peered":"2024-09-15T23:36:23.594592+0000","last_unstale":"2024-09-15T23:36:30.024861+0000","last_undegraded":"2024-09-15T23:36:30.024861+0000","last_fullsized":"2024-09-15T23:36:30.024861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:05.334429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024659+0000","last_change":"2024-09-15T23:36:22.568643+0000","last_active":"2024-09-15T23:36:30.024659+0000","last_peered":"2024-09-15T23:36:30.024659+0000","last_clean":"2024-09-15T23:36:30.024659+0000","last_became_active":"2024-09-15T23:36:22.568351+0000","last_became_peered":"2024-09-15T23:36:22.568351+0000","last_unstale":"2024-09-15T23:36:30.024659+0000","last_undegraded":"2024-09-15T23:36:30.024659+0000","last_fullsized":"2024-09-15T23:36:30.024659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:39.192475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023472+0000","last_change":"2024-09-15T23:36:27.605261+0000","last_active":"2024-09-15T23:36:30.023472+0000","last_peered":"2024-09-15T23:36:30.023472+0000","last_clean":"2024-09-15T23:36:30.023472+0000","last_became_active":"2024-09-15T23:36:27.604780+0000","last_became_peered":"2024-09-15T23:36:27.604780+0000","last_unstale":"2024-09-15T23:36:30.023472+0000","last_undegraded":"2024-09-15T23:36:30.023472+0000","last_fullsized":"2024-09-15T23:36:30.023472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:09.371636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621977+0000","last_change":"2024-09-15T23:36:23.582678+0000","last_active":"2024-09-15T23:36:28.621977+0000","last_peered":"2024-09-15T23:36:28.621977+0000","last_clean":"2024-09-15T23:36:28.621977+0000","last_became_active":"2024-09-15T23:36:23.582022+0000","last_became_peered":"2024-09-15T23:36:23.582022+0000","last_unstale":"2024-09-15T23:36:28.621977+0000","last_undegraded":"2024-09-15T23:36:28.621977+0000","last_fullsized":"2024-09-15T23:36:28.621977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:14.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622315+0000","last_change":"2024-09-15T23:36:22.552625+0000","last_active":"2024-09-15T23:36:28.622315+0000","last_peered":"2024-09-15T23:36:28.622315+0000","last_clean":"2024-09-15T23:36:28.622315+0000","last_became_active":"2024-09-15T23:36:22.552330+0000","last_became_peered":"2024-09-15T23:36:22.552330+0000","last_unstale":"2024-09-15T23:36:28.622315+0000","last_undegraded":"2024-09-15T23:36:28.622315+0000","last_fullsized":"2024-09-15T23:36:28.622315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:06:31.226510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024467+0000","last_change":"2024-09-15T23:36:27.607752+0000","last_active":"2024-09-15T23:36:30.024467+0000","last_peered":"2024-09-15T23:36:30.024467+0000","last_clean":"2024-09-15T23:36:30.024467+0000","last_became_active":"2024-09-15T23:36:27.607269+0000","last_became_peered":"2024-09-15T23:36:27.607269+0000","last_unstale":"2024-09-15T23:36:30.024467+0000","last_undegraded":"2024-09-15T23:36:30.024467+0000","last_fullsized":"2024-09-15T23:36:30.024467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:55.005028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023687+0000","last_change":"2024-09-15T23:36:23.585722+0000","last_active":"2024-09-15T23:36:30.023687+0000","last_peered":"2024-09-15T23:36:30.023687+0000","last_clean":"2024-09-15T23:36:30.023687+0000","last_became_active":"2024-09-15T23:36:23.585435+0000","last_became_peered":"2024-09-15T23:36:23.585435+0000","last_unstale":"2024-09-15T23:36:30.023687+0000","last_undegraded":"2024-09-15T23:36:30.023687+0000","last_fullsized":"2024-09-15T23:36:30.023687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023646+0000","last_change":"2024-09-15T23:36:22.568106+0000","last_active":"2024-09-15T23:36:30.023646+0000","last_peered":"2024-09-15T23:36:30.023646+0000","last_clean":"2024-09-15T23:36:30.023646+0000","last_became_active":"2024-09-15T23:36:22.567770+0000","last_became_peered":"2024-09-15T23:36:22.567770+0000","last_unstale":"2024-09-15T23:36:30.023646+0000","last_undegraded":"2024-09-15T23:36:30.023646+0000","last_fullsized":"2024-09-15T23:36:30.023646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:57.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024431+0000","last_change":"2024-09-15T23:36:27.611234+0000","last_active":"2024-09-15T23:36:30.024431+0000","last_peered":"2024-09-15T23:36:30.024431+0000","last_clean":"2024-09-15T23:36:30.024431+0000","last_became_active":"2024-09-15T23:36:27.610828+0000","last_became_peered":"2024-09-15T23:36:27.610828+0000","last_unstale":"2024-09-15T23:36:30.024431+0000","last_undegraded":"2024-09-15T23:36:30.024431+0000","last_fullsized":"2024-09-15T23:36:30.024431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:30.372516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622286+0000","last_change":"2024-09-15T23:36:23.592816+0000","last_active":"2024-09-15T23:36:28.622286+0000","last_peered":"2024-09-15T23:36:28.622286+0000","last_clean":"2024-09-15T23:36:28.622286+0000","last_became_active":"2024-09-15T23:36:23.590219+0000","last_became_peered":"2024-09-15T23:36:23.590219+0000","last_unstale":"2024-09-15T23:36:28.622286+0000","last_undegraded":"2024-09-15T23:36:28.622286+0000","last_fullsized":"2024-09-15T23:36:28.622286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:09:38.065436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622572+0000","last_change":"2024-09-15T23:36:22.555612+0000","last_active":"2024-09-15T23:36:28.622572+0000","last_peered":"2024-09-15T23:36:28.622572+0000","last_clean":"2024-09-15T23:36:28.622572+0000","last_became_active":"2024-09-15T23:36:22.555279+0000","last_became_peered":"2024-09-15T23:36:22.555279+0000","last_unstale":"2024-09-15T23:36:28.622572+0000","last_undegraded":"2024-09-15T23:36:28.622572+0000","last_fullsized":"2024-09-15T23:36:28.622572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:18.004944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622407+0000","last_change":"2024-09-15T23:36:27.594766+0000","last_active":"2024-09-15T23:36:28.622407+0000","last_peered":"2024-09-15T23:36:28.622407+0000","last_clean":"2024-09-15T23:36:28.622407+0000","last_became_active":"2024-09-15T23:36:27.594179+0000","last_became_peered":"2024-09-15T23:36:27.594179+0000","last_unstale":"2024-09-15T23:36:28.622407+0000","last_undegraded":"2024-09-15T23:36:28.622407+0000","last_fullsized":"2024-09-15T23:36:28.622407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:43.647541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622052+0000","last_change":"2024-09-15T23:36:23.595059+0000","last_active":"2024-09-15T23:36:28.622052+0000","last_peered":"2024-09-15T23:36:28.622052+0000","last_clean":"2024-09-15T23:36:28.622052+0000","last_became_active":"2024-09-15T23:36:23.594760+0000","last_became_peered":"2024-09-15T23:36:23.594760+0000","last_unstale":"2024-09-15T23:36:28.622052+0000","last_undegraded":"2024-09-15T23:36:28.622052+0000","last_fullsized":"2024-09-15T23:36:28.622052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:20.169043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454742+0000","last_change":"2024-09-15T23:36:22.569556+0000","last_active":"2024-09-15T23:36:29.454742+0000","last_peered":"2024-09-15T23:36:29.454742+0000","last_clean":"2024-09-15T23:36:29.454742+0000","last_became_active":"2024-09-15T23:36:22.569214+0000","last_became_peered":"2024-09-15T23:36:22.569214+0000","last_unstale":"2024-09-15T23:36:29.454742+0000","last_undegraded":"2024-09-15T23:36:29.454742+0000","last_fullsized":"2024-09-15T23:36:29.454742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:14:47.610942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622749+0000","last_change":"2024-09-15T23:36:27.601370+0000","last_active":"2024-09-15T23:36:28.622749+0000","last_peered":"2024-09-15T23:36:28.622749+0000","last_clean":"2024-09-15T23:36:28.622749+0000","last_became_active":"2024-09-15T23:36:27.600995+0000","last_became_peered":"2024-09-15T23:36:27.600995+0000","last_unstale":"2024-09-15T23:36:28.622749+0000","last_undegraded":"2024-09-15T23:36:28.622749+0000","last_fullsized":"2024-09-15T23:36:28.622749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:13.579126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025128+0000","last_change":"2024-09-15T23:36:23.595877+0000","last_active":"2024-09-15T23:36:30.025128+0000","last_peered":"2024-09-15T23:36:30.025128+0000","last_clean":"2024-09-15T23:36:30.025128+0000","last_became_active":"2024-09-15T23:36:23.595086+0000","last_became_peered":"2024-09-15T23:36:23.595086+0000","last_unstale":"2024-09-15T23:36:30.025128+0000","last_undegraded":"2024-09-15T23:36:30.025128+0000","last_fullsized":"2024-09-15T23:36:30.025128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:19.739767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:33.124757+0000","last_change":"2024-09-15T23:36:22.551636+0000","last_active":"2024-09-15T23:36:33.124757+0000","last_peered":"2024-09-15T23:36:33.124757+0000","last_clean":"2024-09-15T23:36:33.124757+0000","last_became_active":"2024-09-15T23:36:22.551316+0000","last_became_peered":"2024-09-15T23:36:22.551316+0000","last_unstale":"2024-09-15T23:36:33.124757+0000","last_undegraded":"2024-09-15T23:36:33.124757+0000","last_fullsized":"2024-09-15T23:36:33.124757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624158+0000","last_change":"2024-09-15T23:36:27.606629+0000","last_active":"2024-09-15T23:36:28.624158+0000","last_peered":"2024-09-15T23:36:28.624158+0000","last_clean":"2024-09-15T23:36:28.624158+0000","last_became_active":"2024-09-15T23:36:27.606224+0000","last_became_peered":"2024-09-15T23:36:27.606224+0000","last_unstale":"2024-09-15T23:36:28.624158+0000","last_undegraded":"2024-09-15T23:36:28.624158+0000","last_fullsized":"2024-09-15T23:36:28.624158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:09.576731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622435+0000","last_change":"2024-09-15T23:36:23.580904+0000","last_active":"2024-09-15T23:36:28.622435+0000","last_peered":"2024-09-15T23:36:28.622435+0000","last_clean":"2024-09-15T23:36:28.622435+0000","last_became_active":"2024-09-15T23:36:23.580459+0000","last_became_peered":"2024-09-15T23:36:23.580459+0000","last_unstale":"2024-09-15T23:36:28.622435+0000","last_undegraded":"2024-09-15T23:36:28.622435+0000","last_fullsized":"2024-09-15T23:36:28.622435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:50.078320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454752+0000","last_change":"2024-09-15T23:36:22.557264+0000","last_active":"2024-09-15T23:36:29.454752+0000","last_peered":"2024-09-15T23:36:29.454752+0000","last_clean":"2024-09-15T23:36:29.454752+0000","last_became_active":"2024-09-15T23:36:22.556929+0000","last_became_peered":"2024-09-15T23:36:22.556929+0000","last_unstale":"2024-09-15T23:36:29.454752+0000","last_undegraded":"2024-09-15T23:36:29.454752+0000","last_fullsized":"2024-09-15T23:36:29.454752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:57.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024486+0000","last_change":"2024-09-15T23:36:27.605955+0000","last_active":"2024-09-15T23:36:30.024486+0000","last_peered":"2024-09-15T23:36:30.024486+0000","last_clean":"2024-09-15T23:36:30.024486+0000","last_became_active":"2024-09-15T23:36:27.605521+0000","last_became_peered":"2024-09-15T23:36:27.605521+0000","last_unstale":"2024-09-15T23:36:30.024486+0000","last_undegraded":"2024-09-15T23:36:30.024486+0000","last_fullsized":"2024-09-15T23:36:30.024486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:23.776974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024511+0000","last_change":"2024-09-15T23:36:23.583781+0000","last_active":"2024-09-15T23:36:30.024511+0000","last_peered":"2024-09-15T23:36:30.024511+0000","last_clean":"2024-09-15T23:36:30.024511+0000","last_became_active":"2024-09-15T23:36:23.583475+0000","last_became_peered":"2024-09-15T23:36:23.583475+0000","last_unstale":"2024-09-15T23:36:30.024511+0000","last_undegraded":"2024-09-15T23:36:30.024511+0000","last_fullsized":"2024-09-15T23:36:30.024511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:23.983256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024984+0000","last_change":"2024-09-15T23:36:22.569943+0000","last_active":"2024-09-15T23:36:30.024984+0000","last_peered":"2024-09-15T23:36:30.024984+0000","last_clean":"2024-09-15T23:36:30.024984+0000","last_became_active":"2024-09-15T23:36:22.568617+0000","last_became_peered":"2024-09-15T23:36:22.568617+0000","last_unstale":"2024-09-15T23:36:30.024984+0000","last_undegraded":"2024-09-15T23:36:30.024984+0000","last_fullsized":"2024-09-15T23:36:30.024984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:44.229130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454281+0000","last_change":"2024-09-15T23:36:27.604270+0000","last_active":"2024-09-15T23:36:29.454281+0000","last_peered":"2024-09-15T23:36:29.454281+0000","last_clean":"2024-09-15T23:36:29.454281+0000","last_became_active":"2024-09-15T23:36:27.603416+0000","last_became_peered":"2024-09-15T23:36:27.603416+0000","last_unstale":"2024-09-15T23:36:29.454281+0000","last_undegraded":"2024-09-15T23:36:29.454281+0000","last_fullsized":"2024-09-15T23:36:29.454281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:12.747026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024835+0000","last_change":"2024-09-15T23:36:23.580585+0000","last_active":"2024-09-15T23:36:30.024835+0000","last_peered":"2024-09-15T23:36:30.024835+0000","last_clean":"2024-09-15T23:36:30.024835+0000","last_became_active":"2024-09-15T23:36:23.579003+0000","last_became_peered":"2024-09-15T23:36:23.579003+0000","last_unstale":"2024-09-15T23:36:30.024835+0000","last_undegraded":"2024-09-15T23:36:30.024835+0000","last_fullsized":"2024-09-15T23:36:30.024835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:24.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025091+0000","last_change":"2024-09-15T23:36:22.557388+0000","last_active":"2024-09-15T23:36:30.025091+0000","last_peered":"2024-09-15T23:36:30.025091+0000","last_clean":"2024-09-15T23:36:30.025091+0000","last_became_active":"2024-09-15T23:36:22.556138+0000","last_became_peered":"2024-09-15T23:36:22.556138+0000","last_unstale":"2024-09-15T23:36:30.025091+0000","last_undegraded":"2024-09-15T23:36:30.025091+0000","last_fullsized":"2024-09-15T23:36:30.025091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623557+0000","last_change":"2024-09-15T23:36:27.600449+0000","last_active":"2024-09-15T23:36:28.623557+0000","last_peered":"2024-09-15T23:36:28.623557+0000","last_clean":"2024-09-15T23:36:28.623557+0000","last_became_active":"2024-09-15T23:36:27.599937+0000","last_became_peered":"2024-09-15T23:36:27.599937+0000","last_unstale":"2024-09-15T23:36:28.623557+0000","last_undegraded":"2024-09-15T23:36:28.623557+0000","last_fullsized":"2024-09-15T23:36:28.623557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:13.541381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623785+0000","last_change":"2024-09-15T23:36:27.607667+0000","last_active":"2024-09-15T23:36:28.623785+0000","last_peered":"2024-09-15T23:36:28.623785+0000","last_clean":"2024-09-15T23:36:28.623785+0000","last_became_active":"2024-09-15T23:36:27.607227+0000","last_became_peered":"2024-09-15T23:36:27.607227+0000","last_unstale":"2024-09-15T23:36:28.623785+0000","last_undegraded":"2024-09-15T23:36:28.623785+0000","last_fullsized":"2024-09-15T23:36:28.623785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:33.572386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623619+0000","last_change":"2024-09-15T23:36:23.579014+0000","last_active":"2024-09-15T23:36:28.623619+0000","last_peered":"2024-09-15T23:36:28.623619+0000","last_clean":"2024-09-15T23:36:28.623619+0000","last_became_active":"2024-09-15T23:36:23.578709+0000","last_became_peered":"2024-09-15T23:36:23.578709+0000","last_unstale":"2024-09-15T23:36:28.623619+0000","last_undegraded":"2024-09-15T23:36:28.623619+0000","last_fullsized":"2024-09-15T23:36:28.623619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:07.005224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622050+0000","last_change":"2024-09-15T23:36:22.556649+0000","last_active":"2024-09-15T23:36:28.622050+0000","last_peered":"2024-09-15T23:36:28.622050+0000","last_clean":"2024-09-15T23:36:28.622050+0000","last_became_active":"2024-09-15T23:36:22.556332+0000","last_became_peered":"2024-09-15T23:36:22.556332+0000","last_unstale":"2024-09-15T23:36:28.622050+0000","last_undegraded":"2024-09-15T23:36:28.622050+0000","last_fullsized":"2024-09-15T23:36:28.622050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:46.855766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623003+0000","last_change":"2024-09-15T23:36:27.612447+0000","last_active":"2024-09-15T23:36:28.623003+0000","last_peered":"2024-09-15T23:36:28.623003+0000","last_clean":"2024-09-15T23:36:28.623003+0000","last_became_active":"2024-09-15T23:36:27.612007+0000","last_became_peered":"2024-09-15T23:36:27.612007+0000","last_unstale":"2024-09-15T23:36:28.623003+0000","last_undegraded":"2024-09-15T23:36:28.623003+0000","last_fullsized":"2024-09-15T23:36:28.623003+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:15.233014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023818+0000","last_change":"2024-09-15T23:36:23.592928+0000","last_active":"2024-09-15T23:36:30.023818+0000","last_peered":"2024-09-15T23:36:30.023818+0000","last_clean":"2024-09-15T23:36:30.023818+0000","last_became_active":"2024-09-15T23:36:23.592641+0000","last_became_peered":"2024-09-15T23:36:23.592641+0000","last_unstale":"2024-09-15T23:36:30.023818+0000","last_undegraded":"2024-09-15T23:36:30.023818+0000","last_fullsized":"2024-09-15T23:36:30.023818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:02.943531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622171+0000","last_change":"2024-09-15T23:36:22.569415+0000","last_active":"2024-09-15T23:36:28.622171+0000","last_peered":"2024-09-15T23:36:28.622171+0000","last_clean":"2024-09-15T23:36:28.622171+0000","last_became_active":"2024-09-15T23:36:22.569105+0000","last_became_peered":"2024-09-15T23:36:22.569105+0000","last_unstale":"2024-09-15T23:36:28.622171+0000","last_undegraded":"2024-09-15T23:36:28.622171+0000","last_fullsized":"2024-09-15T23:36:28.622171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:39.526732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622420+0000","last_change":"2024-09-15T23:36:22.569714+0000","last_active":"2024-09-15T23:36:28.622420+0000","last_peered":"2024-09-15T23:36:28.622420+0000","last_clean":"2024-09-15T23:36:28.622420+0000","last_became_active":"2024-09-15T23:36:22.569371+0000","last_became_peered":"2024-09-15T23:36:22.569371+0000","last_unstale":"2024-09-15T23:36:28.622420+0000","last_undegraded":"2024-09-15T23:36:28.622420+0000","last_fullsized":"2024-09-15T23:36:28.622420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:19.167117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622958+0000","last_change":"2024-09-15T23:36:23.591060+0000","last_active":"2024-09-15T23:36:28.622958+0000","last_peered":"2024-09-15T23:36:28.622958+0000","last_clean":"2024-09-15T23:36:28.622958+0000","last_became_active":"2024-09-15T23:36:23.590748+0000","last_became_peered":"2024-09-15T23:36:23.590748+0000","last_unstale":"2024-09-15T23:36:28.622958+0000","last_undegraded":"2024-09-15T23:36:28.622958+0000","last_fullsized":"2024-09-15T23:36:28.622958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:56.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.624452+0000","last_change":"2024-09-15T23:36:27.602410+0000","last_active":"2024-09-15T23:36:28.624452+0000","last_peered":"2024-09-15T23:36:28.624452+0000","last_clean":"2024-09-15T23:36:28.624452+0000","last_became_active":"2024-09-15T23:36:27.602009+0000","last_became_peered":"2024-09-15T23:36:27.602009+0000","last_unstale":"2024-09-15T23:36:28.624452+0000","last_undegraded":"2024-09-15T23:36:28.624452+0000","last_fullsized":"2024-09-15T23:36:28.624452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:38.882900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024165+0000","last_change":"2024-09-15T23:36:22.562409+0000","last_active":"2024-09-15T23:36:30.024165+0000","last_peered":"2024-09-15T23:36:30.024165+0000","last_clean":"2024-09-15T23:36:30.024165+0000","last_became_active":"2024-09-15T23:36:22.561579+0000","last_became_peered":"2024-09-15T23:36:22.561579+0000","last_unstale":"2024-09-15T23:36:30.024165+0000","last_undegraded":"2024-09-15T23:36:30.024165+0000","last_fullsized":"2024-09-15T23:36:30.024165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:21.537792+0000","last_clean_scrub_stamp":"2024-09-15T23:36:21.537792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:46.879331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023841+0000","last_change":"2024-09-15T23:36:23.584956+0000","last_active":"2024-09-15T23:36:30.023841+0000","last_peered":"2024-09-15T23:36:30.023841+0000","last_clean":"2024-09-15T23:36:30.023841+0000","last_became_active":"2024-09-15T23:36:23.584680+0000","last_became_peered":"2024-09-15T23:36:23.584680+0000","last_unstale":"2024-09-15T23:36:30.023841+0000","last_undegraded":"2024-09-15T23:36:30.023841+0000","last_fullsized":"2024-09-15T23:36:30.023841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454641+0000","last_change":"2024-09-15T23:36:27.603930+0000","last_active":"2024-09-15T23:36:29.454641+0000","last_peered":"2024-09-15T23:36:29.454641+0000","last_clean":"2024-09-15T23:36:29.454641+0000","last_became_active":"2024-09-15T23:36:27.603486+0000","last_became_peered":"2024-09-15T23:36:27.603486+0000","last_unstale":"2024-09-15T23:36:29.454641+0000","last_undegraded":"2024-09-15T23:36:29.454641+0000","last_fullsized":"2024-09-15T23:36:29.454641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:26.576924+0000","last_clean_scrub_stamp":"2024-09-15T23:36:26.576924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:41.146991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024615+0000","last_change":"2024-09-15T23:36:23.590531+0000","last_active":"2024-09-15T23:36:30.024615+0000","last_peered":"2024-09-15T23:36:30.024615+0000","last_clean":"2024-09-15T23:36:30.024615+0000","last_became_active":"2024-09-15T23:36:23.590278+0000","last_became_peered":"2024-09-15T23:36:23.590278+0000","last_unstale":"2024-09-15T23:36:30.024615+0000","last_undegraded":"2024-09-15T23:36:30.024615+0000","last_fullsized":"2024-09-15T23:36:30.024615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:39.285245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454276+0000","last_change":"2024-09-15T23:36:23.580382+0000","last_active":"2024-09-15T23:36:29.454276+0000","last_peered":"2024-09-15T23:36:29.454276+0000","last_clean":"2024-09-15T23:36:29.454276+0000","last_became_active":"2024-09-15T23:36:23.580096+0000","last_became_peered":"2024-09-15T23:36:23.580096+0000","last_unstale":"2024-09-15T23:36:29.454276+0000","last_undegraded":"2024-09-15T23:36:29.454276+0000","last_fullsized":"2024-09-15T23:36:29.454276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622200+0000","last_change":"2024-09-15T23:36:23.587488+0000","last_active":"2024-09-15T23:36:28.622200+0000","last_peered":"2024-09-15T23:36:28.622200+0000","last_clean":"2024-09-15T23:36:28.622200+0000","last_became_active":"2024-09-15T23:36:23.587199+0000","last_became_peered":"2024-09-15T23:36:23.587199+0000","last_unstale":"2024-09-15T23:36:28.622200+0000","last_undegraded":"2024-09-15T23:36:28.622200+0000","last_fullsized":"2024-09-15T23:36:28.622200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:41.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023999+0000","last_change":"2024-09-15T23:36:23.583971+0000","last_active":"2024-09-15T23:36:30.023999+0000","last_peered":"2024-09-15T23:36:30.023999+0000","last_clean":"2024-09-15T23:36:30.023999+0000","last_became_active":"2024-09-15T23:36:23.583670+0000","last_became_peered":"2024-09-15T23:36:23.583670+0000","last_unstale":"2024-09-15T23:36:30.023999+0000","last_undegraded":"2024-09-15T23:36:30.023999+0000","last_fullsized":"2024-09-15T23:36:30.023999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:56.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622689+0000","last_change":"2024-09-15T23:36:23.598604+0000","last_active":"2024-09-15T23:36:28.622689+0000","last_peered":"2024-09-15T23:36:28.622689+0000","last_clean":"2024-09-15T23:36:28.622689+0000","last_became_active":"2024-09-15T23:36:23.598283+0000","last_became_peered":"2024-09-15T23:36:23.598283+0000","last_unstale":"2024-09-15T23:36:28.622689+0000","last_undegraded":"2024-09-15T23:36:28.622689+0000","last_fullsized":"2024-09-15T23:36:28.622689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:46.930120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622563+0000","last_change":"2024-09-15T23:36:23.588957+0000","last_active":"2024-09-15T23:36:28.622563+0000","last_peered":"2024-09-15T23:36:28.622563+0000","last_clean":"2024-09-15T23:36:28.622563+0000","last_became_active":"2024-09-15T23:36:23.588651+0000","last_became_peered":"2024-09-15T23:36:23.588651+0000","last_unstale":"2024-09-15T23:36:28.622563+0000","last_undegraded":"2024-09-15T23:36:28.622563+0000","last_fullsized":"2024-09-15T23:36:28.622563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:05.334429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024765+0000","last_change":"2024-09-15T23:36:23.579681+0000","last_active":"2024-09-15T23:36:30.024765+0000","last_peered":"2024-09-15T23:36:30.024765+0000","last_clean":"2024-09-15T23:36:30.024765+0000","last_became_active":"2024-09-15T23:36:23.579299+0000","last_became_peered":"2024-09-15T23:36:23.579299+0000","last_unstale":"2024-09-15T23:36:30.024765+0000","last_undegraded":"2024-09-15T23:36:30.024765+0000","last_fullsized":"2024-09-15T23:36:30.024765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:19.006869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024346+0000","last_change":"2024-09-15T23:36:23.587478+0000","last_active":"2024-09-15T23:36:30.024346+0000","last_peered":"2024-09-15T23:36:30.024346+0000","last_clean":"2024-09-15T23:36:30.024346+0000","last_became_active":"2024-09-15T23:36:23.586993+0000","last_became_peered":"2024-09-15T23:36:23.586993+0000","last_unstale":"2024-09-15T23:36:30.024346+0000","last_undegraded":"2024-09-15T23:36:30.024346+0000","last_fullsized":"2024-09-15T23:36:30.024346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:11:50.246113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622283+0000","last_change":"2024-09-15T23:36:23.594827+0000","last_active":"2024-09-15T23:36:28.622283+0000","last_peered":"2024-09-15T23:36:28.622283+0000","last_clean":"2024-09-15T23:36:28.622283+0000","last_became_active":"2024-09-15T23:36:23.594542+0000","last_became_peered":"2024-09-15T23:36:23.594542+0000","last_unstale":"2024-09-15T23:36:28.622283+0000","last_undegraded":"2024-09-15T23:36:28.622283+0000","last_fullsized":"2024-09-15T23:36:28.622283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.872038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023772+0000","last_change":"2024-09-15T23:36:23.588492+0000","last_active":"2024-09-15T23:36:30.023772+0000","last_peered":"2024-09-15T23:36:30.023772+0000","last_clean":"2024-09-15T23:36:30.023772+0000","last_became_active":"2024-09-15T23:36:23.588154+0000","last_became_peered":"2024-09-15T23:36:23.588154+0000","last_unstale":"2024-09-15T23:36:30.023772+0000","last_undegraded":"2024-09-15T23:36:30.023772+0000","last_fullsized":"2024-09-15T23:36:30.023772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:50:24.116022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023752+0000","last_change":"2024-09-15T23:36:23.584149+0000","last_active":"2024-09-15T23:36:30.023752+0000","last_peered":"2024-09-15T23:36:30.023752+0000","last_clean":"2024-09-15T23:36:30.023752+0000","last_became_active":"2024-09-15T23:36:23.583864+0000","last_became_peered":"2024-09-15T23:36:23.583864+0000","last_unstale":"2024-09-15T23:36:30.023752+0000","last_undegraded":"2024-09-15T23:36:30.023752+0000","last_fullsized":"2024-09-15T23:36:30.023752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:09.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.622823+0000","last_change":"2024-09-15T23:36:23.581142+0000","last_active":"2024-09-15T23:36:28.622823+0000","last_peered":"2024-09-15T23:36:28.622823+0000","last_clean":"2024-09-15T23:36:28.622823+0000","last_became_active":"2024-09-15T23:36:23.580551+0000","last_became_peered":"2024-09-15T23:36:23.580551+0000","last_unstale":"2024-09-15T23:36:28.622823+0000","last_undegraded":"2024-09-15T23:36:28.622823+0000","last_fullsized":"2024-09-15T23:36:28.622823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:44.358074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025231+0000","last_change":"2024-09-15T23:36:23.596339+0000","last_active":"2024-09-15T23:36:30.025231+0000","last_peered":"2024-09-15T23:36:30.025231+0000","last_clean":"2024-09-15T23:36:30.025231+0000","last_became_active":"2024-09-15T23:36:23.590862+0000","last_became_peered":"2024-09-15T23:36:23.590862+0000","last_unstale":"2024-09-15T23:36:30.025231+0000","last_undegraded":"2024-09-15T23:36:30.025231+0000","last_fullsized":"2024-09-15T23:36:30.025231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:06.543007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623049+0000","last_change":"2024-09-15T23:36:23.589509+0000","last_active":"2024-09-15T23:36:28.623049+0000","last_peered":"2024-09-15T23:36:28.623049+0000","last_clean":"2024-09-15T23:36:28.623049+0000","last_became_active":"2024-09-15T23:36:23.589201+0000","last_became_peered":"2024-09-15T23:36:23.589201+0000","last_unstale":"2024-09-15T23:36:28.623049+0000","last_undegraded":"2024-09-15T23:36:28.623049+0000","last_fullsized":"2024-09-15T23:36:28.623049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:02.514262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:29.454968+0000","last_change":"2024-09-15T23:36:23.582505+0000","last_active":"2024-09-15T23:36:29.454968+0000","last_peered":"2024-09-15T23:36:29.454968+0000","last_clean":"2024-09-15T23:36:29.454968+0000","last_became_active":"2024-09-15T23:36:23.582210+0000","last_became_peered":"2024-09-15T23:36:23.582210+0000","last_unstale":"2024-09-15T23:36:29.454968+0000","last_undegraded":"2024-09-15T23:36:29.454968+0000","last_fullsized":"2024-09-15T23:36:29.454968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.623156+0000","last_change":"2024-09-15T23:36:23.590697+0000","last_active":"2024-09-15T23:36:28.623156+0000","last_peered":"2024-09-15T23:36:28.623156+0000","last_clean":"2024-09-15T23:36:28.623156+0000","last_became_active":"2024-09-15T23:36:23.590399+0000","last_became_peered":"2024-09-15T23:36:23.590399+0000","last_unstale":"2024-09-15T23:36:28.623156+0000","last_undegraded":"2024-09-15T23:36:28.623156+0000","last_fullsized":"2024-09-15T23:36:28.623156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.804071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621897+0000","last_change":"2024-09-15T23:36:23.583813+0000","last_active":"2024-09-15T23:36:28.621897+0000","last_peered":"2024-09-15T23:36:28.621897+0000","last_clean":"2024-09-15T23:36:28.621897+0000","last_became_active":"2024-09-15T23:36:23.583516+0000","last_became_peered":"2024-09-15T23:36:23.583516+0000","last_unstale":"2024-09-15T23:36:28.621897+0000","last_undegraded":"2024-09-15T23:36:28.621897+0000","last_fullsized":"2024-09-15T23:36:28.621897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:40.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023592+0000","last_change":"2024-09-15T23:36:23.587846+0000","last_active":"2024-09-15T23:36:30.023592+0000","last_peered":"2024-09-15T23:36:30.023592+0000","last_clean":"2024-09-15T23:36:30.023592+0000","last_became_active":"2024-09-15T23:36:23.587540+0000","last_became_peered":"2024-09-15T23:36:23.587540+0000","last_unstale":"2024-09-15T23:36:30.023592+0000","last_undegraded":"2024-09-15T23:36:30.023592+0000","last_fullsized":"2024-09-15T23:36:30.023592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:31.212365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621934+0000","last_change":"2024-09-15T23:36:23.589402+0000","last_active":"2024-09-15T23:36:28.621934+0000","last_peered":"2024-09-15T23:36:28.621934+0000","last_clean":"2024-09-15T23:36:28.621934+0000","last_became_active":"2024-09-15T23:36:23.589096+0000","last_became_peered":"2024-09-15T23:36:23.589096+0000","last_unstale":"2024-09-15T23:36:28.621934+0000","last_undegraded":"2024-09-15T23:36:28.621934+0000","last_fullsized":"2024-09-15T23:36:28.621934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:19.006869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:28.621955+0000","last_change":"2024-09-15T23:36:23.598740+0000","last_active":"2024-09-15T23:36:28.621955+0000","last_peered":"2024-09-15T23:36:28.621955+0000","last_clean":"2024-09-15T23:36:28.621955+0000","last_became_active":"2024-09-15T23:36:23.598307+0000","last_became_peered":"2024-09-15T23:36:23.598307+0000","last_unstale":"2024-09-15T23:36:28.621955+0000","last_undegraded":"2024-09-15T23:36:28.621955+0000","last_fullsized":"2024-09-15T23:36:28.621955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:24.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025141+0000","last_change":"2024-09-15T23:36:23.591784+0000","last_active":"2024-09-15T23:36:30.025141+0000","last_peered":"2024-09-15T23:36:30.025141+0000","last_clean":"2024-09-15T23:36:30.025141+0000","last_became_active":"2024-09-15T23:36:23.591022+0000","last_became_peered":"2024-09-15T23:36:23.591022+0000","last_unstale":"2024-09-15T23:36:30.025141+0000","last_undegraded":"2024-09-15T23:36:30.025141+0000","last_fullsized":"2024-09-15T23:36:30.025141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:23.323481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025097+0000","last_change":"2024-09-15T23:36:23.586049+0000","last_active":"2024-09-15T23:36:30.025097+0000","last_peered":"2024-09-15T23:36:30.025097+0000","last_clean":"2024-09-15T23:36:30.025097+0000","last_became_active":"2024-09-15T23:36:23.585679+0000","last_became_peered":"2024-09-15T23:36:23.585679+0000","last_unstale":"2024-09-15T23:36:30.025097+0000","last_undegraded":"2024-09-15T23:36:30.025097+0000","last_fullsized":"2024-09-15T23:36:30.025097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:39.610334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024825+0000","last_change":"2024-09-15T23:36:23.595842+0000","last_active":"2024-09-15T23:36:30.024825+0000","last_peered":"2024-09-15T23:36:30.024825+0000","last_clean":"2024-09-15T23:36:30.024825+0000","last_became_active":"2024-09-15T23:36:23.595526+0000","last_became_peered":"2024-09-15T23:36:23.595526+0000","last_unstale":"2024-09-15T23:36:30.024825+0000","last_undegraded":"2024-09-15T23:36:30.024825+0000","last_fullsized":"2024-09-15T23:36:30.024825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:37.109784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.025005+0000","last_change":"2024-09-15T23:36:23.572643+0000","last_active":"2024-09-15T23:36:30.025005+0000","last_peered":"2024-09-15T23:36:30.025005+0000","last_clean":"2024-09-15T23:36:30.025005+0000","last_became_active":"2024-09-15T23:36:23.572340+0000","last_became_peered":"2024-09-15T23:36:23.572340+0000","last_unstale":"2024-09-15T23:36:30.025005+0000","last_undegraded":"2024-09-15T23:36:30.025005+0000","last_fullsized":"2024-09-15T23:36:30.025005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:47.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023623+0000","last_change":"2024-09-15T23:36:23.592206+0000","last_active":"2024-09-15T23:36:30.023623+0000","last_peered":"2024-09-15T23:36:30.023623+0000","last_clean":"2024-09-15T23:36:30.023623+0000","last_became_active":"2024-09-15T23:36:23.590955+0000","last_became_peered":"2024-09-15T23:36:23.590955+0000","last_unstale":"2024-09-15T23:36:30.023623+0000","last_undegraded":"2024-09-15T23:36:30.023623+0000","last_fullsized":"2024-09-15T23:36:30.023623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:17.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023548+0000","last_change":"2024-09-15T23:36:23.594637+0000","last_active":"2024-09-15T23:36:30.023548+0000","last_peered":"2024-09-15T23:36:30.023548+0000","last_clean":"2024-09-15T23:36:30.023548+0000","last_became_active":"2024-09-15T23:36:23.594333+0000","last_became_peered":"2024-09-15T23:36:23.594333+0000","last_unstale":"2024-09-15T23:36:30.023548+0000","last_undegraded":"2024-09-15T23:36:30.023548+0000","last_fullsized":"2024-09-15T23:36:30.023548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:51.033435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024411+0000","last_change":"2024-09-15T23:36:23.592276+0000","last_active":"2024-09-15T23:36:30.024411+0000","last_peered":"2024-09-15T23:36:30.024411+0000","last_clean":"2024-09-15T23:36:30.024411+0000","last_became_active":"2024-09-15T23:36:23.591988+0000","last_became_peered":"2024-09-15T23:36:23.591988+0000","last_unstale":"2024-09-15T23:36:30.024411+0000","last_undegraded":"2024-09-15T23:36:30.024411+0000","last_fullsized":"2024-09-15T23:36:30.024411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:45.231055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024046+0000","last_change":"2024-09-15T23:36:23.596925+0000","last_active":"2024-09-15T23:36:30.024046+0000","last_peered":"2024-09-15T23:36:30.024046+0000","last_clean":"2024-09-15T23:36:30.024046+0000","last_became_active":"2024-09-15T23:36:23.596610+0000","last_became_peered":"2024-09-15T23:36:23.596610+0000","last_unstale":"2024-09-15T23:36:30.024046+0000","last_undegraded":"2024-09-15T23:36:30.024046+0000","last_fullsized":"2024-09-15T23:36:30.024046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:56.256643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.023789+0000","last_change":"2024-09-15T23:36:23.591422+0000","last_active":"2024-09-15T23:36:30.023789+0000","last_peered":"2024-09-15T23:36:30.023789+0000","last_clean":"2024-09-15T23:36:30.023789+0000","last_became_active":"2024-09-15T23:36:23.590380+0000","last_became_peered":"2024-09-15T23:36:23.590380+0000","last_unstale":"2024-09-15T23:36:30.023789+0000","last_undegraded":"2024-09-15T23:36:30.023789+0000","last_fullsized":"2024-09-15T23:36:30.023789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:20.169043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024050+0000","last_change":"2024-09-15T23:36:23.592150+0000","last_active":"2024-09-15T23:36:30.024050+0000","last_peered":"2024-09-15T23:36:30.024050+0000","last_clean":"2024-09-15T23:36:30.024050+0000","last_became_active":"2024-09-15T23:36:23.591205+0000","last_became_peered":"2024-09-15T23:36:23.591205+0000","last_unstale":"2024-09-15T23:36:30.024050+0000","last_undegraded":"2024-09-15T23:36:30.024050+0000","last_fullsized":"2024-09-15T23:36:30.024050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:50.950584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024247+0000","last_change":"2024-09-15T23:36:23.593241+0000","last_active":"2024-09-15T23:36:30.024247+0000","last_peered":"2024-09-15T23:36:30.024247+0000","last_clean":"2024-09-15T23:36:30.024247+0000","last_became_active":"2024-09-15T23:36:23.591733+0000","last_became_peered":"2024-09-15T23:36:23.591733+0000","last_unstale":"2024-09-15T23:36:30.024247+0000","last_undegraded":"2024-09-15T23:36:30.024247+0000","last_fullsized":"2024-09-15T23:36:30.024247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:23.692197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-15T23:36:30.024357+0000","last_change":"2024-09-15T23:36:23.596045+0000","last_active":"2024-09-15T23:36:30.024357+0000","last_peered":"2024-09-15T23:36:30.024357+0000","last_clean":"2024-09-15T23:36:30.024357+0000","last_became_active":"2024-09-15T23:36:23.595078+0000","last_became_peered":"2024-09-15T23:36:23.595078+0000","last_unstale":"2024-09-15T23:36:30.024357+0000","last_undegraded":"2024-09-15T23:36:30.024357+0000","last_fullsized":"2024-09-15T23:36:30.024357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:22.539717+0000","last_clean_scrub_stamp":"2024-09-15T23:36:22.539717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:40.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":262144,"data_stored":103393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":270336,"data_stored":106655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":856064,"data_stored":693249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":102867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":102935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":258048,"data_stored":102811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":860160,"data_stored":693271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":107327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:36:46.969 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T23:36:46.969 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T23:36:46.969 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T23:36:46.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T23:36:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.104+0000 7f01f8778640 1 Processor -- start 2024-09-15T23:36:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.104+0000 7f01f8778640 1 -- start start 2024-09-15T23:36:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f8778640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f00d52e0 0x7f01f006d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f8778640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01f006dbb0 0x7f01f00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f8778640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f00d0a30 con 0x7f01f015e840 2024-09-15T23:36:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f8778640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f01f00cfe90 con 0x7f01f006dbb0 2024-09-15T23:36:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f8778640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f01f00cf560 con 0x7f01f00d52e0 2024-09-15T23:36:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f6d15640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01f006dbb0 0x7f01f00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f6d15640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01f006dbb0 0x7f01f00d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53710/0 (socket says 172.21.15.7:53710) 2024-09-15T23:36:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f6d15640 1 -- 172.21.15.7:0/419574683 learned_addr learned my addr 172.21.15.7:0/419574683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f5d13640 1 --2- 172.21.15.7:0/419574683 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f00d52e0 0x7f01f006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f6d15640 1 -- 172.21.15.7:0/419574683 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f00d52e0 msgr2=0x7f01f006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f6d15640 1 --2- 172.21.15.7:0/419574683 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f00d52e0 0x7f01f006d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f6d15640 1 -- 172.21.15.7:0/419574683 >> v1:172.21.15.18:6789/0 conn(0x7f01f015e840 legacy=0x7f01f00d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.105+0000 7f01f6d15640 1 -- 172.21.15.7:0/419574683 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f00cf840 con 0x7f01f006dbb0 2024-09-15T23:36:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.106+0000 7f01f6d15640 1 --2- 172.21.15.7:0/419574683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 0x7f01f00d0f90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009e50 tx=0x7f01e802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201477ec3f49126c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.106+0000 7f01f5512640 1 -- 172.21.15.7:0/419574683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8036070 con 0x7f01f006dbb0 2024-09-15T23:36:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.106+0000 7f01f5512640 1 -- 172.21.15.7:0/419574683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e8031600 con 0x7f01f006dbb0 2024-09-15T23:36:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.106+0000 7f01f5512640 1 -- 172.21.15.7:0/419574683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80319d0 con 0x7f01f006dbb0 2024-09-15T23:36:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.107+0000 7f01f8778640 1 -- 172.21.15.7:0/419574683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 msgr2=0x7f01f00d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.107+0000 7f01f8778640 1 --2- 172.21.15.7:0/419574683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 0x7f01f00d0f90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009e50 tx=0x7f01e802a710 comp rx=0 tx=0).stop 2024-09-15T23:36:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.107+0000 7f01f8778640 1 -- 172.21.15.7:0/419574683 shutdown_connections 2024-09-15T23:36:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.107+0000 7f01f8778640 1 --2- 172.21.15.7:0/419574683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 0x7f01f00d0f90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.107+0000 7f01f8778640 1 --2- 172.21.15.7:0/419574683 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f00d52e0 0x7f01f006d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.107+0000 7f01f8778640 1 -- 172.21.15.7:0/419574683 >> 172.21.15.7:0/419574683 conn(0x7f01f008bfd0 msgr2=0x7f01f00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.107+0000 7f01f8778640 1 -- 172.21.15.7:0/419574683 shutdown_connections 2024-09-15T23:36:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.107+0000 7f01f8778640 1 -- 172.21.15.7:0/419574683 wait complete. 2024-09-15T23:36:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.108+0000 7f01f8778640 1 Processor -- start 2024-09-15T23:36:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.108+0000 7f01f8778640 1 -- start start 2024-09-15T23:36:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f8778640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 0x7f01f0156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f8778640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f00d52e0 0x7f01f00b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f8778640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f015e840 0x7f01f00b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f8778640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01f015ec20 con 0x7f01f006dbb0 2024-09-15T23:36:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f8778640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01f006e030 con 0x7f01f00d52e0 2024-09-15T23:36:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f8778640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f01f00b6150 con 0x7f01f015e840 2024-09-15T23:36:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f6514640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 0x7f01f0156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f6514640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 0x7f01f0156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53718/0 (socket says 172.21.15.7:53718) 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f6514640 1 -- 172.21.15.7:0/4293811953 learned_addr learned my addr 172.21.15.7:0/4293811953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f6d15640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f015e840 0x7f01f00b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f5d13640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f00d52e0 0x7f01f00b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f6514640 1 -- 172.21.15.7:0/4293811953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f015e840 msgr2=0x7f01f00b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f6514640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f015e840 0x7f01f00b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f6514640 1 -- 172.21.15.7:0/4293811953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f00d52e0 msgr2=0x7f01f00b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f6514640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f00d52e0 0x7f01f00b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.109+0000 7f01f6514640 1 -- 172.21.15.7:0/4293811953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f00b50c0 con 0x7f01f006dbb0 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.110+0000 7f01f6514640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 0x7f01f0156ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c400 tx=0x7f01ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.110+0000 7f01e6ffd640 1 -- 172.21.15.7:0/4293811953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0052d0 con 0x7f01f006dbb0 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.110+0000 7f01e6ffd640 1 -- 172.21.15.7:0/4293811953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec005490 con 0x7f01f006dbb0 2024-09-15T23:36:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.110+0000 7f01e6ffd640 1 -- 172.21.15.7:0/4293811953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec007260 con 0x7f01f006dbb0 2024-09-15T23:36:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.110+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f00cb830 con 0x7f01f006dbb0 2024-09-15T23:36:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.110+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f00c9510 con 0x7f01f006dbb0 2024-09-15T23:36:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.111+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f00d0f90 con 0x7f01f006dbb0 2024-09-15T23:36:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.111+0000 7f01e6ffd640 1 -- 172.21.15.7:0/4293811953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f01ec007400 con 0x7f01f006dbb0 2024-09-15T23:36:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.112+0000 7f01e6ffd640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f01d406c2b0 0x7f01d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.112+0000 7f01f5d13640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f01d406c2b0 0x7f01d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.112+0000 7f01e6ffd640 1 -- 172.21.15.7:0/4293811953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f01ec08abc0 con 0x7f01f006dbb0 2024-09-15T23:36:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.112+0000 7f01f5d13640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f01d406c2b0 0x7f01d406e770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01f00b8350 tx=0x7f01e0009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:47.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.116+0000 7f01e6ffd640 1 -- 172.21.15.7:0/4293811953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f01ec058a30 con 0x7f01f006dbb0 2024-09-15T23:36:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.330+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f01f00d1cb0 con 0x7f01f006dbb0 2024-09-15T23:36:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.331+0000 7f01e6ffd640 1 -- 172.21.15.7:0/4293811953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f01ec05cd30 con 0x7f01f006dbb0 2024-09-15T23:36:47.333 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:47.333 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T23:36:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f01d406c2b0 msgr2=0x7f01d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f01d406c2b0 0x7f01d406e770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01f00b8350 tx=0x7f01e0009110 comp rx=0 tx=0).stop 2024-09-15T23:36:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 msgr2=0x7f01f0156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 0x7f01f0156ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c400 tx=0x7f01ec00c8c0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 shutdown_connections 2024-09-15T23:36:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f01d406c2b0 0x7f01d406e770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01f015e840 0x7f01f00b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f00d52e0 0x7f01f00b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 --2- 172.21.15.7:0/4293811953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f006dbb0 0x7f01f0156ea0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 >> 172.21.15.7:0/4293811953 conn(0x7f01f008bfd0 msgr2=0x7f01f00b7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 shutdown_connections 2024-09-15T23:36:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.333+0000 7f01f8778640 1 -- 172.21.15.7:0/4293811953 wait complete. 2024-09-15T23:36:47.345 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T23:36:47.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:36:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.446+0000 7fa2bc98d640 1 Processor -- start 2024-09-15T23:36:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.446+0000 7fa2bc98d640 1 -- start start 2024-09-15T23:36:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2bc98d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 0x7fa2b806f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2bc98d640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa2b815e5f0 0x7fa2b815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2bc98d640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b80ce270 con 0x7fa2b815aac0 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2bc98d640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b806fd90 con 0x7fa2b815e5f0 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2bc98d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b815efa0 con 0x7fa2b80ceff0 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2b659c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa2b815aac0 0x7fa2b80b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57730/0 (socket says 172.21.15.7:57730) 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2b659c640 1 -- 172.21.15.7:0/3417548167 learned_addr learned my addr 172.21.15.7:0/3417548167 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2b5d9b640 1 --2- 172.21.15.7:0/3417548167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 0x7fa2b806f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2b6d9d640 1 --2- 172.21.15.7:0/3417548167 >> v2:172.21.15.18:3300/0 conn(0x7fa2b815e5f0 0x7fa2b815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2b559a640 1 -- 172.21.15.7:0/3417548167 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933396658 0 0) 0x7fa2b80ce270 con 0x7fa2b815aac0 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.447+0000 7fa2b559a640 1 -- 172.21.15.7:0/3417548167 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2a0003520 con 0x7fa2b815aac0 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.448+0000 7fa2b559a640 1 -- 172.21.15.7:0/3417548167 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 723228704 0 0) 0x7fa2a0003520 con 0x7fa2b815aac0 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.448+0000 7fa2b5d9b640 1 -- 172.21.15.7:0/3417548167 >> v2:172.21.15.18:3300/0 conn(0x7fa2b815e5f0 msgr2=0x7fa2b815ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.448+0000 7fa2b5d9b640 1 --2- 172.21.15.7:0/3417548167 >> v2:172.21.15.18:3300/0 conn(0x7fa2b815e5f0 0x7fa2b815ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.448+0000 7fa2b5d9b640 1 -- 172.21.15.7:0/3417548167 >> v1:172.21.15.7:6789/0 conn(0x7fa2b815aac0 legacy=0x7fa2b80b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.448+0000 7fa2b5d9b640 1 -- 172.21.15.7:0/3417548167 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b80ce510 con 0x7fa2b80ceff0 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.448+0000 7fa2b5d9b640 1 --2- 172.21.15.7:0/3417548167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 0x7fa2b806f7b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0097a0 tx=0x7fa2ac02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6d88802286ac51 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2b559a640 1 -- 172.21.15.7:0/3417548167 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac037070 con 0x7fa2b80ceff0 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2b559a640 1 -- 172.21.15.7:0/3417548167 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ac02aba0 con 0x7fa2b80ceff0 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2b559a640 1 -- 172.21.15.7:0/3417548167 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8001670 con 0x7fa2b80ceff0 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2bc98d640 1 -- 172.21.15.7:0/3417548167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 msgr2=0x7fa2b806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2bc98d640 1 --2- 172.21.15.7:0/3417548167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 0x7fa2b806f7b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0097a0 tx=0x7fa2ac02a110 comp rx=0 tx=0).stop 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2bc98d640 1 -- 172.21.15.7:0/3417548167 shutdown_connections 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2bc98d640 1 --2- 172.21.15.7:0/3417548167 >> v2:172.21.15.18:3300/0 conn(0x7fa2b815e5f0 0x7fa2b815ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2bc98d640 1 --2- 172.21.15.7:0/3417548167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 0x7fa2b806f7b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2bc98d640 1 -- 172.21.15.7:0/3417548167 >> 172.21.15.7:0/3417548167 conn(0x7fa2b80b21e0 msgr2=0x7fa2b80b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2bc98d640 1 -- 172.21.15.7:0/3417548167 shutdown_connections 2024-09-15T23:36:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.449+0000 7fa2bc98d640 1 -- 172.21.15.7:0/3417548167 wait complete. 2024-09-15T23:36:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2bc98d640 1 Processor -- start 2024-09-15T23:36:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2bc98d640 1 -- start start 2024-09-15T23:36:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2bc98d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 0x7fa2b80aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2bc98d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b815aac0 0x7fa2b815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2bc98d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b815e5f0 0x7fa2b81540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2bc98d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b815aea0 con 0x7fa2b815aac0 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2bc98d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b80cead0 con 0x7fa2b815e5f0 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2bc98d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b806df00 con 0x7fa2b80ceff0 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2b5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b815aac0 0x7fa2b815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2b5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b815aac0 0x7fa2b815c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53728/0 (socket says 172.21.15.7:53728) 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.451+0000 7fa2b5d9b640 1 -- 172.21.15.7:0/568236444 learned_addr learned my addr 172.21.15.7:0/568236444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2b6d9d640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b815e5f0 0x7fa2b81540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2b659c640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 0x7fa2b80aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2b5d9b640 1 -- 172.21.15.7:0/568236444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 msgr2=0x7fa2b80aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2b5d9b640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 0x7fa2b80aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2b5d9b640 1 -- 172.21.15.7:0/568236444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b815e5f0 msgr2=0x7fa2b81540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2b5d9b640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b815e5f0 0x7fa2b81540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2b5d9b640 1 -- 172.21.15.7:0/568236444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b80b0d20 con 0x7fa2b815aac0 2024-09-15T23:36:47.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2b5d9b640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b815aac0 0x7fa2b815c070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac035040 tx=0x7fa2ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2a6ffd640 1 -- 172.21.15.7:0/568236444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac037050 con 0x7fa2b815aac0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2a6ffd640 1 -- 172.21.15.7:0/568236444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ac043070 con 0x7fa2b815aac0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2a6ffd640 1 -- 172.21.15.7:0/568236444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac006e10 con 0x7fa2b815aac0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b809c860 con 0x7fa2b815aac0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.452+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b809ce20 con 0x7fa2b815aac0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.453+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b806f7b0 con 0x7fa2b815aac0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.454+0000 7fa2a6ffd640 1 -- 172.21.15.7:0/568236444 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa2ac03ca10 con 0x7fa2b815aac0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.454+0000 7fa2a6ffd640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa29806c2e0 0x7fa29806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.454+0000 7fa2a6ffd640 1 -- 172.21.15.7:0/568236444 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa2ac0abfe0 con 0x7fa2b815aac0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.454+0000 7fa2b659c640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa29806c2e0 0x7fa29806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.455+0000 7fa2b659c640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa29806c2e0 0x7fa29806e7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8000f60 tx=0x7fa2a8001660 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.458+0000 7fa2a6ffd640 1 -- 172.21.15.7:0/568236444 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa2ac079f70 con 0x7fa2b815aac0 2024-09-15T23:36:47.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.670+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2b80cf9f0 con 0x7fa2b815aac0 2024-09-15T23:36:47.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.671+0000 7fa2a6ffd640 1 -- 172.21.15.7:0/568236444 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5210 (secure 0 0 0) 0x7fa2ac07e270 con 0x7fa2b815aac0 2024-09-15T23:36:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:47.673 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-15T23:36:42:878666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:42.233279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:36:47.673 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-15T23:36:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa29806c2e0 msgr2=0x7fa29806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa29806c2e0 0x7fa29806e7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8000f60 tx=0x7fa2a8001660 comp rx=0 tx=0).stop 2024-09-15T23:36:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b815aac0 msgr2=0x7fa2b815c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b815aac0 0x7fa2b815c070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac035040 tx=0x7fa2ac0046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 shutdown_connections 2024-09-15T23:36:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b815e5f0 0x7fa2b81540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b815aac0 0x7fa2b815c070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa29806c2e0 0x7fa29806e7a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 --2- 172.21.15.7:0/568236444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2b80ceff0 0x7fa2b80aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 >> 172.21.15.7:0/568236444 conn(0x7fa2b80b21e0 msgr2=0x7fa2b80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 shutdown_connections 2024-09-15T23:36:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.673+0000 7fa2bc98d640 1 -- 172.21.15.7:0/568236444 wait complete. 2024-09-15T23:36:47.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:36:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.785+0000 7f04d00d7640 1 Processor -- start 2024-09-15T23:36:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.786+0000 7f04d00d7640 1 -- start start 2024-09-15T23:36:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.786+0000 7f04d00d7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c80b1c00 0x7f04c8169670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.786+0000 7f04d00d7640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c80d1390 con 0x7f04c80b31c0 2024-09-15T23:36:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.786+0000 7f04d00d7640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c80b2030 con 0x7f04c80b2de0 2024-09-15T23:36:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.786+0000 7f04d00d7640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f04c806cd20 con 0x7f04c80b1c00 2024-09-15T23:36:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.786+0000 7f04cd672640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f04c80b31c0 0x7f04c806dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57744/0 (socket says 172.21.15.7:57744) 2024-09-15T23:36:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cd672640 1 -- 172.21.15.7:0/3090890302 learned_addr learned my addr 172.21.15.7:0/3090890302 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04ce674640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c80b1c00 0x7f04c8169670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3780175349 0 0) 0x7f04c80d1390 con 0x7f04c80b31c0 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04b0003520 con 0x7f04c80b31c0 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1222853331 0 0) 0x7f04b0003520 con 0x7f04c80b31c0 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c80b1c00 msgr2=0x7f04c8169670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c80b1c00 0x7f04c8169670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 >> v1:172.21.15.18:6789/0 conn(0x7f04c80b2de0 legacy=0x7f04c806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c80d1630 con 0x7f04c80b31c0 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f04bc001670 con 0x7f04c80b31c0 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 >> v1:172.21.15.7:6789/0 conn(0x7f04c80b31c0 legacy=0x7f04c806dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04b00039e0 0x7f04b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04b0004800 0x7f04b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c80b1c00 0x7f04b0005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04c806cd20 con 0x7f04b00039e0 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f04b000bb50 con 0x7f04b0004800 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f04b000bd50 con 0x7f04c80b1c00 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cde73640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04b00039e0 0x7f04b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04cd672640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04b0004800 0x7f04b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.787+0000 7f04ce674640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c80b1c00 0x7f04b0005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.788+0000 7f04cde73640 1 -- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c80b1c00 msgr2=0x7f04b0005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.788+0000 7f04cde73640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c80b1c00 0x7f04b0005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.788+0000 7f04cde73640 1 -- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04b0004800 msgr2=0x7f04b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.788+0000 7f04cde73640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04b0004800 0x7f04b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.788+0000 7f04cde73640 1 -- 172.21.15.7:0/3090890302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b0003520 con 0x7f04b00039e0 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.788+0000 7f04cde73640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04b00039e0 0x7f04b0003e00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f04b8001750 tx=0x7f04b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc8b4e4a4963dec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.788+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8001370 con 0x7f04b00039e0 2024-09-15T23:36:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.788+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b80052d0 con 0x7f04b00039e0 2024-09-15T23:36:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.788+0000 7f04cce71640 1 -- 172.21.15.7:0/3090890302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b802abe0 con 0x7f04b00039e0 2024-09-15T23:36:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.789+0000 7f04d00d7640 1 -- 172.21.15.7:0/3090890302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04b00039e0 msgr2=0x7f04b0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.789+0000 7f04d00d7640 1 --2- 172.21.15.7:0/3090890302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04b00039e0 0x7f04b0003e00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f04b8001750 tx=0x7f04b0005830 comp rx=0 tx=0).stop 2024-09-15T23:36:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.789+0000 7f04ce674640 1 -- 172.21.15.7:0/3090890302 reap_dead start 2024-09-15T23:36:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.789+0000 7f04d00d7640 1 -- 172.21.15.7:0/3090890302 shutdown_connections 2024-09-15T23:36:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.789+0000 7f04d00d7640 1 -- 172.21.15.7:0/3090890302 >> 172.21.15.7:0/3090890302 conn(0x7f04c80b3f00 msgr2=0x7f04c80b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.789+0000 7f04d00d7640 1 -- 172.21.15.7:0/3090890302 shutdown_connections 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.789+0000 7f04d00d7640 1 -- 172.21.15.7:0/3090890302 wait complete. 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.790+0000 7f04d00d7640 1 Processor -- start 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.790+0000 7f04d00d7640 1 -- start start 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04d00d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04b00039e0 0x7f04c815f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04d00d7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c8166d30 0x7f04c80b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04d00d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c80b7b30 0x7f04c80b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04d00d7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04c80b3970 con 0x7f04c80b7b30 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04d00d7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f04c806cf70 con 0x7f04b00039e0 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04d00d7640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f04c8167660 con 0x7f04c8166d30 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04ce674640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c80b7b30 0x7f04c80b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04ce674640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c80b7b30 0x7f04c80b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53764/0 (socket says 172.21.15.7:53764) 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04ce674640 1 -- 172.21.15.7:0/2578172262 learned_addr learned my addr 172.21.15.7:0/2578172262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04cde73640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04b00039e0 0x7f04c815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.791+0000 7f04cd672640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c8166d30 0x7f04c80b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04ce674640 1 -- 172.21.15.7:0/2578172262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c8166d30 msgr2=0x7f04c80b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04ce674640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c8166d30 0x7f04c80b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04ce674640 1 -- 172.21.15.7:0/2578172262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04b00039e0 msgr2=0x7f04c815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04ce674640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04b00039e0 0x7f04c815f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04ce674640 1 -- 172.21.15.7:0/2578172262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c815ff30 con 0x7f04c80b7b30 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04ce674640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c80b7b30 0x7f04c80b5890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f04c0011c80 tx=0x7f04c000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04c67fc640 1 -- 172.21.15.7:0/2578172262 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0015430 con 0x7f04c80b7b30 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04c67fc640 1 -- 172.21.15.7:0/2578172262 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c001d070 con 0x7f04c80b7b30 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04c67fc640 1 -- 172.21.15.7:0/2578172262 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0015830 con 0x7f04c80b7b30 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c8160210 con 0x7f04c80b7b30 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.792+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c80b5370 con 0x7f04c80b7b30 2024-09-15T23:36:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.793+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c806dc50 con 0x7f04c80b7b30 2024-09-15T23:36:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.794+0000 7f04c67fc640 1 -- 172.21.15.7:0/2578172262 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f04c00159d0 con 0x7f04c80b7b30 2024-09-15T23:36:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.794+0000 7f04c67fc640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f04b406c2b0 0x7f04b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.794+0000 7f04c67fc640 1 -- 172.21.15.7:0/2578172262 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f04c008bed0 con 0x7f04c80b7b30 2024-09-15T23:36:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.794+0000 7f04cde73640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f04b406c2b0 0x7f04b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.795+0000 7f04cde73640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f04b406c2b0 0x7f04b406e770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04b8001750 tx=0x7f04b802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:47.798+0000 7f04c67fc640 1 -- 172.21.15.7:0/2578172262 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f04c0010040 con 0x7f04c80b7b30 2024-09-15T23:36:48.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.010+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04c80b4ab0 con 0x7f04c80b7b30 2024-09-15T23:36:48.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.011+0000 7f04c67fc640 1 -- 172.21.15.7:0/2578172262 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5210 (secure 0 0 0) 0x7f04c0059cb0 con 0x7f04c80b7b30 2024-09-15T23:36:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-15T23:36:42:878666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:42.233279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:36:48.013 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.012+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f04b406c2b0 msgr2=0x7f04b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.012+0000 7f04d00d7640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f04b406c2b0 0x7f04b406e770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04b8001750 tx=0x7f04b802e000 comp rx=0 tx=0).stop 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c80b7b30 msgr2=0x7f04c80b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c80b7b30 0x7f04c80b5890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f04c0011c80 tx=0x7f04c000f580 comp rx=0 tx=0).stop 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 shutdown_connections 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c80b7b30 0x7f04c80b5890 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f04b406c2b0 0x7f04b406e770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04c8166d30 0x7f04c80b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 --2- 172.21.15.7:0/2578172262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04b00039e0 0x7f04c815f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 >> 172.21.15.7:0/2578172262 conn(0x7f04c80b3f00 msgr2=0x7f04c8161950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 shutdown_connections 2024-09-15T23:36:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.013+0000 7f04d00d7640 1 -- 172.21.15.7:0/2578172262 wait complete. 2024-09-15T23:36:48.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.126+0000 7fa76b8e4640 1 Processor -- start 2024-09-15T23:36:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.126+0000 7fa76b8e4640 1 -- start start 2024-09-15T23:36:48.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa76b8e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa76415e840 0x7fa7640d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa76b8e4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa7640d52e0 0x7fa76406d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa76b8e4640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7640cf560 con 0x7fa76406dbb0 2024-09-15T23:36:48.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa76b8e4640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa7640cfe90 con 0x7fa7640d52e0 2024-09-15T23:36:48.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa76b8e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa7640cf840 con 0x7fa76415e840 2024-09-15T23:36:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa768e7f640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa7640d52e0 0x7fa76406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa769680640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa76415e840 0x7fa7640d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa769e81640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fa76406dbb0 0x7fa7640d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:49746/0 (socket says 172.21.15.7:49746) 2024-09-15T23:36:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa768e7f640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa7640d52e0 0x7fa76406d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:44080/0 (socket says 172.21.15.7:44080) 2024-09-15T23:36:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa769e81640 1 -- 172.21.15.7:0/4097338471 learned_addr learned my addr 172.21.15.7:0/4097338471 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa769680640 1 -- 172.21.15.7:0/4097338471 >> v2:172.21.15.18:3300/0 conn(0x7fa7640d52e0 msgr2=0x7fa76406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa769680640 1 --2- 172.21.15.7:0/4097338471 >> v2:172.21.15.18:3300/0 conn(0x7fa7640d52e0 0x7fa76406d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa769680640 1 -- 172.21.15.7:0/4097338471 >> v1:172.21.15.18:6789/0 conn(0x7fa76406dbb0 legacy=0x7fa7640d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa769680640 1 -- 172.21.15.7:0/4097338471 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7640cfb20 con 0x7fa76415e840 2024-09-15T23:36:48.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.127+0000 7fa768e7f640 1 --2- 172.21.15.7:0/4097338471 >> v2:172.21.15.18:3300/0 conn(0x7fa7640d52e0 0x7fa76406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:48.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.128+0000 7fa769680640 1 --2- 172.21.15.7:0/4097338471 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa76415e840 0x7fa7640d0320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa7600097a0 tx=0x7fa76002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a53e78d0bbf7902 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:48.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.128+0000 7fa75bfff640 1 -- 172.21.15.7:0/4097338471 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760037070 con 0x7fa76415e840 2024-09-15T23:36:48.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.128+0000 7fa75bfff640 1 -- 172.21.15.7:0/4097338471 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa760032440 con 0x7fa76415e840 2024-09-15T23:36:48.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.128+0000 7fa75bfff640 1 -- 172.21.15.7:0/4097338471 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760032810 con 0x7fa76415e840 2024-09-15T23:36:48.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.129+0000 7fa76b8e4640 1 -- 172.21.15.7:0/4097338471 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa76415e840 msgr2=0x7fa7640d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.129+0000 7fa76b8e4640 1 --2- 172.21.15.7:0/4097338471 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa76415e840 0x7fa7640d0320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa7600097a0 tx=0x7fa76002a180 comp rx=0 tx=0).stop 2024-09-15T23:36:48.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.129+0000 7fa76b8e4640 1 -- 172.21.15.7:0/4097338471 shutdown_connections 2024-09-15T23:36:48.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.129+0000 7fa76b8e4640 1 --2- 172.21.15.7:0/4097338471 >> v2:172.21.15.18:3300/0 conn(0x7fa7640d52e0 0x7fa76406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.129+0000 7fa76b8e4640 1 --2- 172.21.15.7:0/4097338471 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa76415e840 0x7fa7640d0320 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.129+0000 7fa76b8e4640 1 -- 172.21.15.7:0/4097338471 >> 172.21.15.7:0/4097338471 conn(0x7fa76408bfd0 msgr2=0x7fa7640d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:48.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.129+0000 7fa76b8e4640 1 -- 172.21.15.7:0/4097338471 shutdown_connections 2024-09-15T23:36:48.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.129+0000 7fa76b8e4640 1 -- 172.21.15.7:0/4097338471 wait complete. 2024-09-15T23:36:48.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.130+0000 7fa76b8e4640 1 Processor -- start 2024-09-15T23:36:48.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.130+0000 7fa76b8e4640 1 -- start start 2024-09-15T23:36:48.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa76b8e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76406dbb0 0x7fa764156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa76b8e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa7640d52e0 0x7fa7641e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa76b8e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa76415e840 0x7fa7641e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa76b8e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa76415ec20 con 0x7fa76406dbb0 2024-09-15T23:36:48.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa76b8e4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa7640d0860 con 0x7fa76415e840 2024-09-15T23:36:48.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa76b8e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa7640ce5f0 con 0x7fa7640d52e0 2024-09-15T23:36:48.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769680640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76406dbb0 0x7fa764156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769680640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76406dbb0 0x7fa764156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53774/0 (socket says 172.21.15.7:53774) 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769680640 1 -- 172.21.15.7:0/3597321870 learned_addr learned my addr 172.21.15.7:0/3597321870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa768e7f640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa7640d52e0 0x7fa7641e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769e81640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa76415e840 0x7fa7641e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769680640 1 -- 172.21.15.7:0/3597321870 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa7640d52e0 msgr2=0x7fa7641e5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769680640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa7640d52e0 0x7fa7641e5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769680640 1 -- 172.21.15.7:0/3597321870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa76415e840 msgr2=0x7fa7641e6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769680640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa76415e840 0x7fa7641e6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769680640 1 -- 172.21.15.7:0/3597321870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7640cda70 con 0x7fa76406dbb0 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.131+0000 7fa769e81640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa76415e840 0x7fa7641e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.132+0000 7fa769680640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76406dbb0 0x7fa764156bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa7600098c0 tx=0x7fa7600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.132+0000 7fa759ffb640 1 -- 172.21.15.7:0/3597321870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760037050 con 0x7fa76406dbb0 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.132+0000 7fa759ffb640 1 -- 172.21.15.7:0/3597321870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa760038040 con 0x7fa76406dbb0 2024-09-15T23:36:48.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.132+0000 7fa759ffb640 1 -- 172.21.15.7:0/3597321870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760030b10 con 0x7fa76406dbb0 2024-09-15T23:36:48.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.132+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7640cdd50 con 0x7fa76406dbb0 2024-09-15T23:36:48.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.132+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7640ce310 con 0x7fa76406dbb0 2024-09-15T23:36:48.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.133+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76406d670 con 0x7fa76406dbb0 2024-09-15T23:36:48.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.133+0000 7fa759ffb640 1 -- 172.21.15.7:0/3597321870 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa760030cb0 con 0x7fa76406dbb0 2024-09-15T23:36:48.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.134+0000 7fa759ffb640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa74806c2b0 0x7fa74806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.134+0000 7fa759ffb640 1 -- 172.21.15.7:0/3597321870 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa7600ac110 con 0x7fa76406dbb0 2024-09-15T23:36:48.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.134+0000 7fa768e7f640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa74806c2b0 0x7fa74806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.134+0000 7fa768e7f640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa74806c2b0 0x7fa74806e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa754009500 tx=0x7fa754008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:48.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.138+0000 7fa759ffb640 1 -- 172.21.15.7:0/3597321870 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa7600313a0 con 0x7fa76406dbb0 2024-09-15T23:36:48.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.315+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7640d1cb0 con 0x7fa76406dbb0 2024-09-15T23:36:48.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.317+0000 7fa759ffb640 1 -- 172.21.15.7:0/3597321870 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7fa76007a5d0 con 0x7fa76406dbb0 2024-09-15T23:36:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:48.319 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa74806c2b0 msgr2=0x7fa74806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa74806c2b0 0x7fa74806e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa754009500 tx=0x7fa754008040 comp rx=0 tx=0).stop 2024-09-15T23:36:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76406dbb0 msgr2=0x7fa764156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76406dbb0 0x7fa764156bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa7600098c0 tx=0x7fa7600046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 shutdown_connections 2024-09-15T23:36:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa76415e840 0x7fa7641e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa74806c2b0 0x7fa74806e770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa7640d52e0 0x7fa7641e5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 --2- 172.21.15.7:0/3597321870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76406dbb0 0x7fa764156bb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 >> 172.21.15.7:0/3597321870 conn(0x7fa76408bfd0 msgr2=0x7fa7640d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:48.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 shutdown_connections 2024-09-15T23:36:48.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.319+0000 7fa76b8e4640 1 -- 172.21.15.7:0/3597321870 wait complete. 2024-09-15T23:36:48.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:36:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.432+0000 7f1b7b247640 1 Processor -- start 2024-09-15T23:36:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.432+0000 7f1b7b247640 1 -- start start 2024-09-15T23:36:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b7b247640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1b740d51c0 0x7f1b740d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b7b247640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b740d30d0 0x7f1b740b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b7b247640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b740b3d10 con 0x7f1b7415e6d0 2024-09-15T23:36:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b7b247640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1b740d28c0 con 0x7f1b740d30d0 2024-09-15T23:36:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b7b247640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f1b740d3500 con 0x7f1b740d51c0 2024-09-15T23:36:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b797e4640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b740d30d0 0x7f1b740b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b78fe3640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1b7415e6d0 0x7f1b740d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57750/0 (socket says 172.21.15.7:57750) 2024-09-15T23:36:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b797e4640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b740d30d0 0x7f1b740b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53776/0 (socket says 172.21.15.7:53776) 2024-09-15T23:36:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b78fe3640 1 -- 172.21.15.7:0/1515488697 learned_addr learned my addr 172.21.15.7:0/1515488697 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b73fff640 1 --2- 172.21.15.7:0/1515488697 >> v2:172.21.15.18:3300/0 conn(0x7f1b740d51c0 0x7f1b740d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b797e4640 1 -- 172.21.15.7:0/1515488697 >> v2:172.21.15.18:3300/0 conn(0x7f1b740d51c0 msgr2=0x7f1b740d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b797e4640 1 --2- 172.21.15.7:0/1515488697 >> v2:172.21.15.18:3300/0 conn(0x7f1b740d51c0 0x7f1b740d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b797e4640 1 -- 172.21.15.7:0/1515488697 >> v1:172.21.15.7:6789/0 conn(0x7f1b7415e6d0 legacy=0x7f1b740d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.433+0000 7f1b797e4640 1 -- 172.21.15.7:0/1515488697 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b740cfcd0 con 0x7f1b740d30d0 2024-09-15T23:36:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.434+0000 7f1b797e4640 1 --2- 172.21.15.7:0/1515488697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d30d0 0x7f1b740b41c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1b7408c600 tx=0x7f1b6c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0758b67085ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.434+0000 7f1b737fe640 1 -- 172.21.15.7:0/1515488697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c007740 con 0x7f1b740d30d0 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.434+0000 7f1b737fe640 1 -- 172.21.15.7:0/1515488697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c0078e0 con 0x7f1b740d30d0 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.434+0000 7f1b737fe640 1 -- 172.21.15.7:0/1515488697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c007cb0 con 0x7f1b740d30d0 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.435+0000 7f1b7b247640 1 -- 172.21.15.7:0/1515488697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d30d0 msgr2=0x7f1b740b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.435+0000 7f1b7b247640 1 --2- 172.21.15.7:0/1515488697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d30d0 0x7f1b740b41c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1b7408c600 tx=0x7f1b6c02a6a0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.435+0000 7f1b7b247640 1 -- 172.21.15.7:0/1515488697 shutdown_connections 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.435+0000 7f1b7b247640 1 --2- 172.21.15.7:0/1515488697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d30d0 0x7f1b740b41c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.435+0000 7f1b7b247640 1 --2- 172.21.15.7:0/1515488697 >> v2:172.21.15.18:3300/0 conn(0x7f1b740d51c0 0x7f1b740d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.435+0000 7f1b7b247640 1 -- 172.21.15.7:0/1515488697 >> 172.21.15.7:0/1515488697 conn(0x7f1b7408bfd0 msgr2=0x7f1b740d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.435+0000 7f1b7b247640 1 -- 172.21.15.7:0/1515488697 shutdown_connections 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.435+0000 7f1b7b247640 1 -- 172.21.15.7:0/1515488697 wait complete. 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.436+0000 7f1b7b247640 1 Processor -- start 2024-09-15T23:36:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.436+0000 7f1b7b247640 1 -- start start 2024-09-15T23:36:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b7b247640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b740d30d0 0x7f1b7415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b7b247640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d51c0 0x7f1b740b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b7b247640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b7415e6d0 0x7f1b74161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b7b247640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7415eab0 con 0x7f1b740d51c0 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b7b247640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1b740d0230 con 0x7f1b7415e6d0 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b7b247640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f1b740d0510 con 0x7f1b740d30d0 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d51c0 0x7f1b740b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d51c0 0x7f1b740b9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53778/0 (socket says 172.21.15.7:53778) 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b73fff640 1 -- 172.21.15.7:0/1786182995 learned_addr learned my addr 172.21.15.7:0/1786182995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b797e4640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b7415e6d0 0x7f1b74161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b78fe3640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b740d30d0 0x7f1b7415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b73fff640 1 -- 172.21.15.7:0/1786182995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b740d30d0 msgr2=0x7f1b7415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b73fff640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b740d30d0 0x7f1b7415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b73fff640 1 -- 172.21.15.7:0/1786182995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b7415e6d0 msgr2=0x7f1b74161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b73fff640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b7415e6d0 0x7f1b74161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b73fff640 1 -- 172.21.15.7:0/1786182995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b741e6d30 con 0x7f1b740d51c0 2024-09-15T23:36:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b797e4640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b7415e6d0 0x7f1b74161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b73fff640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d51c0 0x7f1b740b9c20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c7f0 tx=0x7f1b6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b717fa640 1 -- 172.21.15.7:0/1786182995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60015070 con 0x7f1b740d51c0 2024-09-15T23:36:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.437+0000 7f1b717fa640 1 -- 172.21.15.7:0/1786182995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b60004220 con 0x7f1b740d51c0 2024-09-15T23:36:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.438+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b741e7010 con 0x7f1b740d51c0 2024-09-15T23:36:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.438+0000 7f1b717fa640 1 -- 172.21.15.7:0/1786182995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60007620 con 0x7f1b740d51c0 2024-09-15T23:36:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.438+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b740b7590 con 0x7f1b740d51c0 2024-09-15T23:36:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.438+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b740d2b90 con 0x7f1b740d51c0 2024-09-15T23:36:48.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.439+0000 7f1b717fa640 1 -- 172.21.15.7:0/1786182995 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1b600079f0 con 0x7f1b740d51c0 2024-09-15T23:36:48.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.439+0000 7f1b717fa640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1b5806c2e0 0x7f1b5806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:48.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.440+0000 7f1b717fa640 1 -- 172.21.15.7:0/1786182995 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1b6008b7b0 con 0x7f1b740d51c0 2024-09-15T23:36:48.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.440+0000 7f1b78fe3640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1b5806c2e0 0x7f1b5806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:48.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.440+0000 7f1b78fe3640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1b5806c2e0 0x7f1b5806e7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b68000f60 tx=0x7f1b68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.444+0000 7f1b717fa640 1 -- 172.21.15.7:0/1786182995 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1b60059bd0 con 0x7f1b740d51c0 2024-09-15T23:36:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.653+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b741ea830 con 0x7f1b740d51c0 2024-09-15T23:36:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.653+0000 7f1b717fa640 1 -- 172.21.15.7:0/1786182995 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5210 (secure 0 0 0) 0x7f1b600594d0 con 0x7f1b740d51c0 2024-09-15T23:36:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:48.655 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-15T23:36:42:878666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:42.233279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:36:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-15T23:36:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1b5806c2e0 msgr2=0x7f1b5806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1b5806c2e0 0x7f1b5806e7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b68000f60 tx=0x7f1b68009040 comp rx=0 tx=0).stop 2024-09-15T23:36:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d51c0 msgr2=0x7f1b740b9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d51c0 0x7f1b740b9c20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c7f0 tx=0x7f1b6000ccb0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 shutdown_connections 2024-09-15T23:36:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b7415e6d0 0x7f1b74161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740d51c0 0x7f1b740b9c20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1b5806c2e0 0x7f1b5806e7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 --2- 172.21.15.7:0/1786182995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b740d30d0 0x7f1b7415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 >> 172.21.15.7:0/1786182995 conn(0x7f1b7408bfd0 msgr2=0x7f1b740d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 shutdown_connections 2024-09-15T23:36:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:48.655+0000 7f1b7b247640 1 -- 172.21.15.7:0/1786182995 wait complete. 2024-09-15T23:36:49.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-15T23:36:22.916234+0000', 'modified': '2024-09-15T23:36:42.233279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.18:6833/3035612480', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6832', 'nonce': 3035612480}, {'type': 'v1', 'addr': '172.21.15.18:6833', 'nonce': 3035612480}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4291, 'qdb_cluster': [4291]} 2024-09-15T23:36:49.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-15T23:36:49.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-15T23:36:49.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.177+0000 7f44a6dfd640 1 Processor -- start 2024-09-15T23:36:49.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.178+0000 7f44a6dfd640 1 -- start start 2024-09-15T23:36:49.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.178+0000 7f44a6dfd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f44a015e5b0 0x7f44a0162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.178+0000 7f44a6dfd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44a00d24e0 0x7f44a0162890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.178+0000 7f44a6dfd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e990 0x7f44a0162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.178+0000 7f44a6dfd640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f44a00d2980 con 0x7f44a00d24e0 2024-09-15T23:36:49.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.178+0000 7f44a6dfd640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f44a00d2c40 con 0x7f44a015e5b0 2024-09-15T23:36:49.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.178+0000 7f44a6dfd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f44a00d2f20 con 0x7f44a015e990 2024-09-15T23:36:49.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f449ffff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44a00d24e0 0x7f44a0162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f449ffff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44a00d24e0 0x7f44a0162890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53780/0 (socket says 172.21.15.7:53780) 2024-09-15T23:36:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f449ffff640 1 -- 172.21.15.7:0/358728230 learned_addr learned my addr 172.21.15.7:0/358728230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f44a4b99640 1 --2- 172.21.15.7:0/358728230 >> v2:172.21.15.18:3300/0 conn(0x7f44a015e5b0 0x7f44a0162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f44a539a640 1 --2- 172.21.15.7:0/358728230 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e990 0x7f44a0162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f449ffff640 1 -- 172.21.15.7:0/358728230 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e990 msgr2=0x7f44a0162dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f449ffff640 1 --2- 172.21.15.7:0/358728230 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e990 0x7f44a0162dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f449ffff640 1 -- 172.21.15.7:0/358728230 >> v2:172.21.15.18:3300/0 conn(0x7f44a015e5b0 msgr2=0x7f44a0162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f449ffff640 1 --2- 172.21.15.7:0/358728230 >> v2:172.21.15.18:3300/0 conn(0x7f44a015e5b0 0x7f44a0162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.179+0000 7f449ffff640 1 -- 172.21.15.7:0/358728230 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a00d3200 con 0x7f44a00d24e0 2024-09-15T23:36:49.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f449ffff640 1 --2- 172.21.15.7:0/358728230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 0x7f44a0162890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4494009850 tx=0x7f449402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183f8ef20885827a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f449f7fe640 1 -- 172.21.15.7:0/358728230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494004030 con 0x7f44a00d24e0 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f449f7fe640 1 -- 172.21.15.7:0/358728230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44940041d0 con 0x7f44a00d24e0 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f449f7fe640 1 -- 172.21.15.7:0/358728230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494003610 con 0x7f44a00d24e0 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f44a6dfd640 1 -- 172.21.15.7:0/358728230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 msgr2=0x7f44a0162890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/358728230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 0x7f44a0162890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4494009850 tx=0x7f449402a1e0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f44a6dfd640 1 -- 172.21.15.7:0/358728230 shutdown_connections 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/358728230 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e990 0x7f44a0162dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/358728230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 0x7f44a0162890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/358728230 >> v2:172.21.15.18:3300/0 conn(0x7f44a015e5b0 0x7f44a0162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.180+0000 7f44a6dfd640 1 -- 172.21.15.7:0/358728230 >> 172.21.15.7:0/358728230 conn(0x7f44a008bfd0 msgr2=0x7f44a00d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.181+0000 7f44a6dfd640 1 -- 172.21.15.7:0/358728230 shutdown_connections 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.181+0000 7f44a6dfd640 1 -- 172.21.15.7:0/358728230 wait complete. 2024-09-15T23:36:49.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.182+0000 7f44a6dfd640 1 Processor -- start 2024-09-15T23:36:49.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.182+0000 7f44a6dfd640 1 -- start start 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a6dfd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 0x7f44a00b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a6dfd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e5b0 0x7f44a00c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a6dfd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44a015e990 0x7f44a00b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a6dfd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44a00b3b40 con 0x7f44a00d24e0 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a6dfd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f44a00b39c0 con 0x7f44a015e990 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a6dfd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f44a00d0fa0 con 0x7f44a015e5b0 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a4b99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 0x7f44a00b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a4b99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 0x7f44a00b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53788/0 (socket says 172.21.15.7:53788) 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a4b99640 1 -- 172.21.15.7:0/1021180353 learned_addr learned my addr 172.21.15.7:0/1021180353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f449ffff640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e5b0 0x7f44a00c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a539a640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44a015e990 0x7f44a00b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a4b99640 1 -- 172.21.15.7:0/1021180353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e5b0 msgr2=0x7f44a00c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a4b99640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e5b0 0x7f44a00c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a4b99640 1 -- 172.21.15.7:0/1021180353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44a015e990 msgr2=0x7f44a00b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a4b99640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44a015e990 0x7f44a00b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a4b99640 1 -- 172.21.15.7:0/1021180353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a00b94e0 con 0x7f44a00d24e0 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a539a640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44a015e990 0x7f44a00b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a4b99640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 0x7f44a00b6a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f449000d800 tx=0x7f449000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f449d7fa640 1 -- 172.21.15.7:0/1021180353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449000b6c0 con 0x7f44a00d24e0 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f449d7fa640 1 -- 172.21.15.7:0/1021180353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4490019070 con 0x7f44a00d24e0 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a6dfd640 1 -- 172.21.15.7:0/1021180353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a00b9780 con 0x7f44a00d24e0 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f449d7fa640 1 -- 172.21.15.7:0/1021180353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449000bac0 con 0x7f44a00d24e0 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.183+0000 7f44a6dfd640 1 -- 172.21.15.7:0/1021180353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a00b9a20 con 0x7f44a00d24e0 2024-09-15T23:36:49.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.185+0000 7f449d7fa640 1 -- 172.21.15.7:0/1021180353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f449000bc60 con 0x7f44a00d24e0 2024-09-15T23:36:49.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.185+0000 7f449d7fa640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f448c06c2b0 0x7f448c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.186+0000 7f449d7fa640 1 -- 172.21.15.7:0/1021180353 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f449008bba0 con 0x7f44a00d24e0 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.186+0000 7f449ffff640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f448c06c2b0 0x7f448c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.186+0000 7f449ffff640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f448c06c2b0 0x7f448c06e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4494033000 tx=0x7f449402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f44a9c7a740 1 Processor -- start 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f44a9c7a740 1 -- start start 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa49da80 0x55bdaa4a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 0x55bdaa49a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bdaa402520 con 0x55bdaa49da80 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bdaa3aa3e0 con 0x55bdaa4a02c0 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x55bdaa3fe4e0 con 0x55bdaa4a1a50 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f449ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa49da80 0x55bdaa4a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f449ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa49da80 0x55bdaa4a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53794/0 (socket says 172.21.15.7:53794) 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f449ffff640 1 -- 172.21.15.7:0/100379614 learned_addr learned my addr 172.21.15.7:0/100379614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.187+0000 7f44a4b99640 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f44a539a640 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 0x55bdaa49a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f449ffff640 1 -- 172.21.15.7:0/100379614 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 msgr2=0x55bdaa49a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f449ffff640 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 0x55bdaa49a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f449ffff640 1 -- 172.21.15.7:0/100379614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 msgr2=0x55bdaa4e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f449ffff640 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f449ffff640 1 -- 172.21.15.7:0/100379614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdaa3c52f0 con 0x55bdaa49da80 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f44a4b99640 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f449ffff640 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa49da80 0x55bdaa4a06a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4494004370 tx=0x7f449402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8934986b9711bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f44867fc640 1 -- 172.21.15.7:0/100379614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494004460 con 0x55bdaa49da80 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f44867fc640 1 -- 172.21.15.7:0/100379614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449404e070 con 0x55bdaa49da80 2024-09-15T23:36:49.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.188+0000 7f44867fc640 1 -- 172.21.15.7:0/100379614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494003810 con 0x55bdaa49da80 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 -- 172.21.15.7:0/100379614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa49da80 msgr2=0x55bdaa4a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa49da80 0x55bdaa4a06a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4494004370 tx=0x7f449402c000 comp rx=0 tx=0).stop 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 -- 172.21.15.7:0/100379614 shutdown_connections 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 0x55bdaa49a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa49da80 0x55bdaa4a06a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/100379614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 -- 172.21.15.7:0/100379614 >> 172.21.15.7:0/100379614 conn(0x55bdaa49ca80 msgr2=0x55bdaa4e4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 -- 172.21.15.7:0/100379614 shutdown_connections 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 -- 172.21.15.7:0/100379614 wait complete. 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.189+0000 7f44a9c7a740 1 -- start start 2024-09-15T23:36:49.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.190+0000 7f44a9c7a740 1 Processor -- start 2024-09-15T23:36:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.190+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.190+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 0x55bdaa49b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.190+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa49a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.190+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bdaa49bf20 con 0x55bdaa4c09e0 2024-09-15T23:36:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.190+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bdaa38b450 con 0x55bdaa4a02c0 2024-09-15T23:36:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.190+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x55bdaa4e6290 con 0x55bdaa4a1a50 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.190+0000 7f44a9c7a740 10 client.0 _ll_register_callbacks cb 0x55bdaa114ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a539a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa49a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a539a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa49a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53808/0 (socket says 172.21.15.7:53808) 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a539a640 1 -- 172.21.15.7:0/2816354168 learned_addr learned my addr 172.21.15.7:0/2816354168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a4b99640 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f449ffff640 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 0x55bdaa49b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a539a640 1 -- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 msgr2=0x55bdaa49b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a539a640 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 0x55bdaa49b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a539a640 1 -- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 msgr2=0x55bdaa4dec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a539a640 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4dec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a539a640 1 -- 172.21.15.7:0/2816354168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdaa1152d0 con 0x55bdaa4c09e0 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f449ffff640 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 0x55bdaa49b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a4b99640 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4dec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a539a640 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa49a430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4498007d30 tx=0x7f4498012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf2e3833ea4ab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f447f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f447f7fe640 1 -- 172.21.15.7:0/2816354168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449801c020 con 0x55bdaa4c09e0 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f447f7fe640 1 -- 172.21.15.7:0/2816354168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4498018470 con 0x55bdaa4c09e0 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a9c7a740 1 -- 172.21.15.7:0/2816354168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449404e070 con 0x55bdaa4c09e0 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f447f7fe640 1 -- 172.21.15.7:0/2816354168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f4498026050 con 0x55bdaa4c09e0 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a9c7a740 10 client.4644 fetch_fsmap learned FSMap version 8 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.191+0000 7f44a9c7a740 1 -- 172.21.15.7:0/2816354168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55bdaa1df6b0 con 0x55bdaa4c09e0 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f447f7fe640 1 -- 172.21.15.7:0/2816354168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449404e070 con 0x55bdaa4c09e0 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f447f7fe640 1 -- 172.21.15.7:0/2816354168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f4498029020 con 0x55bdaa4c09e0 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f44a9c7a740 10 client.4644 fetch_fsmap finished waiting for FSMap version 8 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f44a9c7a740 10 client.4644 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f44a9c7a740 20 client.4644 populate_metadata read hostname 'smithi007' 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55bdaa49aaf0 0x55bdaa4c4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f44a9c7a740 4 client.4644 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f44a9c7a740 1 -- 172.21.15.7:0/2816354168 --> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdaa40fc50 con 0x55bdaa49aaf0 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f44a4b99640 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55bdaa49aaf0 0x55bdaa4c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f44a4b99640 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55bdaa49aaf0 0x55bdaa4c4080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=794dc03dd9dacf0 server_cookie=20c74b71f537883a in_seq=0 out_seq=0 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.192+0000 7f447f7fe640 10 client.4644 ms_handle_connect on v2:172.21.15.18:6832/3035612480 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f447f7fe640 1 -- 172.21.15.7:0/2816354168 <== mds.0 v2:172.21.15.18:6832/3035612480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdaa40fc50 con 0x55bdaa49aaf0 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f447f7fe640 10 client.4644 handle_command_reply: tid=0 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 client.4644 shutdown 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 -- 172.21.15.7:0/2816354168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 msgr2=0x55bdaa49a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa49a430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4498007d30 tx=0x7f4498012610 comp rx=0 tx=0).stop 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 -- 172.21.15.7:0/2816354168 shutdown_connections 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55bdaa49aaf0 0x55bdaa4c4080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa49a430 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a1a50 0x55bdaa49b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f447f7fe640 0 client.4644 ms_handle_reset on v2:172.21.15.18:6832/3035612480 2024-09-15T23:36:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/2816354168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4dec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 -- 172.21.15.7:0/2816354168 >> 172.21.15.7:0/2816354168 conn(0x55bdaa49da80 msgr2=0x55bdaa49e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 -- 172.21.15.7:0/2816354168 shutdown_connections 2024-09-15T23:36:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 1 -- 172.21.15.7:0/2816354168 wait complete. 2024-09-15T23:36:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.193+0000 7f44a9c7a740 20 client.4644 trim_cache size 0 max 16384 2024-09-15T23:36:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.211+0000 7f44a9c7a740 1 Processor -- start 2024-09-15T23:36:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.211+0000 7f44a9c7a740 1 -- start start 2024-09-15T23:36:49.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a02c0 0x55bdaa4ded80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 0x55bdaa4e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa4e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0201fc0 con 0x55bdaa4c09e0 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0201040 con 0x55bdaa4a1a50 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f44a02003a0 con 0x55bdaa4a02c0 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a539a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa4e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a539a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa4e3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53812/0 (socket says 172.21.15.7:53812) 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a539a640 1 -- 172.21.15.7:0/4184350915 learned_addr learned my addr 172.21.15.7:0/4184350915 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f449ffff640 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 0x55bdaa4e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a4b99640 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a02c0 0x55bdaa4ded80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a539a640 1 -- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a02c0 msgr2=0x55bdaa4ded80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a539a640 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a02c0 0x55bdaa4ded80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a539a640 1 -- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 msgr2=0x55bdaa4e2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a539a640 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 0x55bdaa4e2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a539a640 1 -- 172.21.15.7:0/4184350915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdaa4c7e50 con 0x55bdaa4c09e0 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f449ffff640 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 0x55bdaa4e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.212+0000 7f44a4b99640 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a02c0 0x55bdaa4ded80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a539a640 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa4e3420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4498009af0 tx=0x7f4498018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9924c60495f7f36e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f4485ffb640 1 -- 172.21.15.7:0/4184350915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449802f070 con 0x55bdaa4c09e0 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f4485ffb640 1 -- 172.21.15.7:0/4184350915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4498021610 con 0x55bdaa4c09e0 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f4485ffb640 1 -- 172.21.15.7:0/4184350915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449801c070 con 0x55bdaa4c09e0 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4184350915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 msgr2=0x55bdaa4e3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa4e3420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4498009af0 tx=0x7f4498018000 comp rx=0 tx=0).stop 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4184350915 shutdown_connections 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4c09e0 0x55bdaa4e3420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 0x55bdaa4e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4184350915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4a02c0 0x55bdaa4ded80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4184350915 >> 172.21.15.7:0/4184350915 conn(0x55bdaa49da80 msgr2=0x55bdaa4e6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:49.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4184350915 shutdown_connections 2024-09-15T23:36:49.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.213+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4184350915 wait complete. 2024-09-15T23:36:49.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.214+0000 7f44a9c7a740 1 -- start start 2024-09-15T23:36:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.214+0000 7f44a9c7a740 1 Processor -- start 2024-09-15T23:36:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 0x55bdaa508c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a9c7a740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4c09e0 0x55bdaa561ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bdaa49a970 con 0x55bdaa4a02c0 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f44a02009f0 con 0x55bdaa4a1a50 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a9c7a740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x55bdaa4c91e0 con 0x55bdaa4c09e0 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a9c7a740 10 client.0 _ll_register_callbacks cb 0x7f4498009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a4b99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a4b99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53822/0 (socket says 172.21.15.7:53822) 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a4b99640 1 -- 172.21.15.7:0/4233929932 learned_addr learned my addr 172.21.15.7:0/4233929932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f449ffff640 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 0x55bdaa508c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a539a640 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4c09e0 0x55bdaa561ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a4b99640 1 -- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4c09e0 msgr2=0x55bdaa561ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a4b99640 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4c09e0 0x55bdaa561ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a4b99640 1 -- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 msgr2=0x55bdaa508c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a4b99640 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 0x55bdaa508c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a4b99640 1 -- 172.21.15.7:0/4233929932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdaa5084b0 con 0x55bdaa4a02c0 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.215+0000 7f44a539a640 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4c09e0 0x55bdaa561ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a4b99640 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4c4a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f449000b4b0 tx=0x7f44900624b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2804ae51f965c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f447f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f447f7fe640 1 -- 172.21.15.7:0/4233929932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490087020 con 0x55bdaa4a02c0 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f447f7fe640 1 -- 172.21.15.7:0/4233929932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4490055f40 con 0x55bdaa4a02c0 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4233929932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4498021610 con 0x55bdaa4a02c0 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f447f7fe640 1 -- 172.21.15.7:0/4233929932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f4490071b30 con 0x55bdaa4a02c0 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a9c7a740 10 client.4650 fetch_fsmap learned FSMap version 8 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4233929932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55bdaa49a240 con 0x55bdaa4a02c0 2024-09-15T23:36:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f447f7fe640 1 -- 172.21.15.7:0/4233929932 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4498021610 con 0x55bdaa4a02c0 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f447f7fe640 1 -- 172.21.15.7:0/4233929932 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f4490013070 con 0x55bdaa4a02c0 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a9c7a740 10 client.4650 fetch_fsmap finished waiting for FSMap version 8 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a9c7a740 10 client.4650 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a9c7a740 20 client.4650 populate_metadata read hostname 'smithi007' 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55bdaa4f4620 0x55bdaa4c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a9c7a740 4 client.4650 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4233929932 --> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] -- command(tid 0: {"prefix": "status"}) -- 0x55bdaa409fa0 con 0x55bdaa4f4620 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.216+0000 7f449ffff640 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55bdaa4f4620 0x55bdaa4c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f447f7fe640 10 client.4650 ms_handle_connect on v2:172.21.15.18:6832/3035612480 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f449ffff640 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55bdaa4f4620 0x55bdaa4c5fa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a99b35c5758e82 server_cookie=273de3d4837a447a in_seq=0 out_seq=0 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f447f7fe640 1 -- 172.21.15.7:0/4233929932 <== mds.0 v2:172.21.15.18:6832/3035612480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55bdaa409fa0 con 0x55bdaa4f4620 2024-09-15T23:36:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f447f7fe640 10 client.4650 handle_command_reply: tid=0 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 client.4650 shutdown 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4233929932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4a02c0 msgr2=0x55bdaa4c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4c4a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f449000b4b0 tx=0x7f44900624b0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4233929932 shutdown_connections 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:6832/3035612480,v1:172.21.15.18:6833/3035612480] conn(0x55bdaa4f4620 0x55bdaa4c5fa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x55bdaa4c09e0 0x55bdaa561ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bdaa4a1a50 0x55bdaa508c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 --2- 172.21.15.7:0/4233929932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdaa4a02c0 0x55bdaa4c4a10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f447f7fe640 0 client.4650 ms_handle_reset on v2:172.21.15.18:6832/3035612480 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4233929932 >> 172.21.15.7:0/4233929932 conn(0x55bdaa49da80 msgr2=0x55bdaa49aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4233929932 shutdown_connections 2024-09-15T23:36:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 1 -- 172.21.15.7:0/4233929932 wait complete. 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.217+0000 7f44a9c7a740 20 client.4650 trim_cache size 0 max 16384 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9da25f4c-4673-468c-a40c-f01392dffc9b", 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4291, 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 25.654401273000001, 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 8, 2024-09-15T23:36:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 25, 2024-09-15T23:36:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 25, 2024-09-15T23:36:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 31.673212622000001 2024-09-15T23:36:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 -- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f448c06c2b0 msgr2=0x7f448c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f448c06c2b0 0x7f448c06e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4494033000 tx=0x7f449402e000 comp rx=0 tx=0).stop 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 -- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 msgr2=0x7f44a00b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 0x7f44a00b6a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f449000d800 tx=0x7f449000dcc0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 -- 172.21.15.7:0/1021180353 shutdown_connections 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44a015e990 0x7f44a00b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f448c06c2b0 0x7f448c06e770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44a015e5b0 0x7f44a00c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 --2- 172.21.15.7:0/1021180353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44a00d24e0 0x7f44a00b6a30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 -- 172.21.15.7:0/1021180353 >> 172.21.15.7:0/1021180353 conn(0x7f44a008bfd0 msgr2=0x7f44a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 -- 172.21.15.7:0/1021180353 shutdown_connections 2024-09-15T23:36:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.218+0000 7f44a6dfd640 1 -- 172.21.15.7:0/1021180353 wait complete. 2024-09-15T23:36:49.231 INFO:teuthology.run_tasks:Running task sequential... 2024-09-15T23:36:49.239 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-15T23:36:49.239 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-15T23:36:49.240 INFO:teuthology.task.print:Enabling mgr modules 2024-09-15T23:36:49.240 INFO:teuthology.run_tasks:Running task kclient... 2024-09-15T23:36:49.267 INFO:tasks.kclient:Mounting kernel clients... 2024-09-15T23:36:49.267 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-15T23:36:49.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:49.267 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-15T23:36:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-15T23:36:49.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:49.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T23:36:49.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:49.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-15T23:36:49.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:49.444 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-15T23:36:49.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:49.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-15T23:36:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-09-15T23:36:49.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:36:49.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:49.532 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-09-15T23:36:49.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:49.552 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-09-15T23:36:49.625 INFO:teuthology.orchestra.run.smithi018.stderr:Cannot find device "ceph-brx" 2024-09-15T23:36:49.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:36:49.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:49.627 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-09-15T23:36:49.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:49.686 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-09-15T23:36:49.755 INFO:teuthology.orchestra.run.smithi018.stderr:Cannot find device "ceph-brx" 2024-09-15T23:36:49.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:36:49.758 INFO:tasks.kclient:client.0 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v1'} 2024-09-15T23:36:49.758 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T23:36:49.758 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T23:36:49.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-15T23:36:49.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.861+0000 7f6aedc8d640 1 Processor -- start 2024-09-15T23:36:49.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.861+0000 7f6aedc8d640 1 -- start start 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.862+0000 7f6aedc8d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae8158e60 0x7f6ae80b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.862+0000 7f6aedc8d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ae80b31c0 0x7f6ae80d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.862+0000 7f6aedc8d640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6ae80b3bc0 0x7f6ae80d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.862+0000 7f6aedc8d640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae806dfa0 con 0x7f6ae80b31c0 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.862+0000 7f6aedc8d640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae8159240 con 0x7f6ae80b3bc0 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.862+0000 7f6aedc8d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae806e210 con 0x7f6ae8158e60 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.862+0000 7f6ae6ffd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ae80b31c0 0x7f6ae80d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.862+0000 7f6ae6ffd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ae80b31c0 0x7f6ae80d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53826/0 (socket says 172.21.15.7:53826) 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.862+0000 7f6ae6ffd640 1 -- 172.21.15.7:0/21775739 learned_addr learned my addr 172.21.15.7:0/21775739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.863+0000 7f6ae77fe640 1 --2- 172.21.15.7:0/21775739 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae8158e60 0x7f6ae80b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.863+0000 7f6ae7fff640 1 --2- 172.21.15.7:0/21775739 >> v2:172.21.15.18:3300/0 conn(0x7f6ae80b3bc0 0x7f6ae80d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.863+0000 7f6ae6ffd640 1 -- 172.21.15.7:0/21775739 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae8158e60 msgr2=0x7f6ae80b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.863+0000 7f6ae6ffd640 1 --2- 172.21.15.7:0/21775739 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae8158e60 0x7f6ae80b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.863+0000 7f6ae6ffd640 1 -- 172.21.15.7:0/21775739 >> v2:172.21.15.18:3300/0 conn(0x7f6ae80b3bc0 msgr2=0x7f6ae80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.863+0000 7f6ae6ffd640 1 --2- 172.21.15.7:0/21775739 >> v2:172.21.15.18:3300/0 conn(0x7f6ae80b3bc0 0x7f6ae80d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.863+0000 7f6ae6ffd640 1 -- 172.21.15.7:0/21775739 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae806e4f0 con 0x7f6ae80b31c0 2024-09-15T23:36:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.863+0000 7f6ae6ffd640 1 --2- 172.21.15.7:0/21775739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae80b31c0 0x7f6ae80d05f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80cfbe0 tx=0x7f6adc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a27841a283a5b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6ae67fc640 1 -- 172.21.15.7:0/21775739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc037070 con 0x7f6ae80b31c0 2024-09-15T23:36:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6ae67fc640 1 -- 172.21.15.7:0/21775739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6adc032440 con 0x7f6ae80b31c0 2024-09-15T23:36:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6ae67fc640 1 -- 172.21.15.7:0/21775739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc032810 con 0x7f6ae80b31c0 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6aedc8d640 1 -- 172.21.15.7:0/21775739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae80b31c0 msgr2=0x7f6ae80d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/21775739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae80b31c0 0x7f6ae80d05f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80cfbe0 tx=0x7f6adc02a270 comp rx=0 tx=0).stop 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6aedc8d640 1 -- 172.21.15.7:0/21775739 shutdown_connections 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/21775739 >> v2:172.21.15.18:3300/0 conn(0x7f6ae80b3bc0 0x7f6ae80d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/21775739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae80b31c0 0x7f6ae80d05f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/21775739 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae8158e60 0x7f6ae80b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6aedc8d640 1 -- 172.21.15.7:0/21775739 >> 172.21.15.7:0/21775739 conn(0x7f6ae80b21e0 msgr2=0x7f6ae80b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6aedc8d640 1 -- 172.21.15.7:0/21775739 shutdown_connections 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.864+0000 7f6aedc8d640 1 -- 172.21.15.7:0/21775739 wait complete. 2024-09-15T23:36:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.865+0000 7f6aedc8d640 1 Processor -- start 2024-09-15T23:36:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.866+0000 7f6aedc8d640 1 -- start start 2024-09-15T23:36:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.866+0000 7f6aedc8d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae80b31c0 0x7f6ae81f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.866+0000 7f6aedc8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae80b3bc0 0x7f6ae809b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.866+0000 7f6aedc8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae8158e60 0x7f6ae809b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.866+0000 7f6aedc8d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae815ca20 con 0x7f6ae8158e60 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.866+0000 7f6aedc8d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae806eab0 con 0x7f6ae80b3bc0 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.866+0000 7f6aedc8d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae815cba0 con 0x7f6ae80b31c0 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae8158e60 0x7f6ae809b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae80b31c0 0x7f6ae81f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae8158e60 0x7f6ae809b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53832/0 (socket says 172.21.15.7:53832) 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae7fff640 1 -- 172.21.15.7:0/1657684097 learned_addr learned my addr 172.21.15.7:0/1657684097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae6ffd640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae80b3bc0 0x7f6ae809b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae7fff640 1 -- 172.21.15.7:0/1657684097 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae80b31c0 msgr2=0x7f6ae81f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae7fff640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae80b31c0 0x7f6ae81f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae7fff640 1 -- 172.21.15.7:0/1657684097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae80b3bc0 msgr2=0x7f6ae809b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae7fff640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae80b3bc0 0x7f6ae809b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae7fff640 1 -- 172.21.15.7:0/1657684097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae81518b0 con 0x7f6ae8158e60 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae77fe640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae80b31c0 0x7f6ae81f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae6ffd640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae80b3bc0 0x7f6ae809b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6ae7fff640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae8158e60 0x7f6ae809b680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009dd0 tx=0x7f6ae000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6acffff640 1 -- 172.21.15.7:0/1657684097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0018020 con 0x7f6ae8158e60 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6acffff640 1 -- 172.21.15.7:0/1657684097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae0014480 con 0x7f6ae8158e60 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6acffff640 1 -- 172.21.15.7:0/1657684097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0014870 con 0x7f6ae8158e60 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae8151b90 con 0x7f6ae8158e60 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.867+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae815da00 con 0x7f6ae8158e60 2024-09-15T23:36:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.868+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae80b3680 con 0x7f6ae8158e60 2024-09-15T23:36:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.869+0000 7f6acffff640 1 -- 172.21.15.7:0/1657684097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6ae001b020 con 0x7f6ae8158e60 2024-09-15T23:36:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.869+0000 7f6acffff640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6ad406c2b0 0x7f6ad406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.869+0000 7f6ae77fe640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6ad406c2b0 0x7f6ad406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.869+0000 7f6acffff640 1 -- 172.21.15.7:0/1657684097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6ae008c1e0 con 0x7f6ae8158e60 2024-09-15T23:36:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.870+0000 7f6ae77fe640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6ad406c2b0 0x7f6ad406e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009eb0 tx=0x7f6ad8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:49.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:49.874+0000 7f6acffff640 1 -- 172.21.15.7:0/1657684097 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6ae0022030 con 0x7f6ae8158e60 2024-09-15T23:36:50.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.085+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6ae80ce790 con 0x7f6ae8158e60 2024-09-15T23:36:50.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.086+0000 7f6acffff640 1 -- 172.21.15.7:0/1657684097 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f6ae0010040 con 0x7f6ae8158e60 2024-09-15T23:36:50.087 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T23:36:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6ad406c2b0 msgr2=0x7f6ad406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6ad406c2b0 0x7f6ad406e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009eb0 tx=0x7f6ad8009190 comp rx=0 tx=0).stop 2024-09-15T23:36:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae8158e60 msgr2=0x7f6ae809b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae8158e60 0x7f6ae809b680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009dd0 tx=0x7f6ae000c490 comp rx=0 tx=0).stop 2024-09-15T23:36:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 shutdown_connections 2024-09-15T23:36:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae8158e60 0x7f6ae809b680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae80b3bc0 0x7f6ae809b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6ad406c2b0 0x7f6ad406e770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 --2- 172.21.15.7:0/1657684097 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ae80b31c0 0x7f6ae81f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 >> 172.21.15.7:0/1657684097 conn(0x7f6ae80b21e0 msgr2=0x7f6ae815bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 shutdown_connections 2024-09-15T23:36:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50.088+0000 7f6aedc8d640 1 -- 172.21.15.7:0/1657684097 wait complete. 2024-09-15T23:36:50.100 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-15T23:36:50.100 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T23:36:50.100 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-15T23:36:50.100 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-15T23:36:50.101 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T23:36:50.101 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-15T23:36:50.101 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T23:36:50.101 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T23:36:50.101 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-15T23:36:50.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:50.101 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-15T23:36:50.117 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:36:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:36:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:36:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T23:36:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42745sec preferred_lft 42745sec 2024-09-15T23:36:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-15T23:36:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:50.119 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-15T23:36:50.119 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T23:36:50.119 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:50.119 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:50.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-09-15T23:36:50.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-09-15T23:36:50.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-09-15T23:36:50.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T23:36:50.120 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:50.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:50.259 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T23:36:50.283 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-15T23:36:50.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:50.285 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-15T23:36:50.343 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-15T23:36:50.343 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T23:36:50.343 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T23:36:50.343 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T23:36:50.343 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T23:36:50.343 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:50.344 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:50.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T23:36:50.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T23:36:50.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T23:36:50.344 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:50.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:50.471 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-15T23:36:50.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:50.487 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-09-15T23:36:50.543 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:50.544 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:50.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T23:36:50.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-15T23:36:50.544 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:50.662 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-15T23:36:50.663 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:50.663 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:50.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-15T23:36:50.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T23:36:50.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-15T23:36:50.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-15T23:36:50.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-15T23:36:50.663 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:50.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:50.780 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:50.780 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:50.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-15T23:36:50.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T23:36:50.780 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:50.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:50.859 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-15T23:36:50.874 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-15T23:36:50.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:50.875 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-15T23:36:50.930 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-15T23:36:50.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:50.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-09-15T23:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-09-15T23:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-09-15T23:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-15T23:36:51.153 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-09-15T23:36:51.154 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-09-15T23:36:51.154 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-15T23:36:51.154 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:36:51.154 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:36:51.171 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-09-15T23:36:51.171 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-09-15T23:36:51.171 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-15T23:36:51.171 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:36:51.171 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:36:51.172 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790" 2024-09-15T23:36:51.172 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790:/ 2024-09-15T23:36:51.172 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs,key=0,fsid=9da25f4c-4673-468c-a40c-f01392dffc9b" will pass to kernel 2024-09-15T23:36:51.172 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-15T23:36:51.172 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-15T23:36:51.172 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-15T23:36:51.172 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-15T23:36:51.172 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-15T23:36:51.174 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T23:36:51.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:51.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-15T23:36:51.199 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:51.297 INFO:teuthology.orchestra.run.smithi007.stdout:client4659 2024-09-15T23:36:51.298 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:36:51.298 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:36:51.314 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:51.412 INFO:teuthology.orchestra.run.smithi007.stdout:client4659 2024-09-15T23:36:51.412 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:51.470 INFO:teuthology.orchestra.run.smithi007.stdout:client4659 2024-09-15T23:36:51.470 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9da25f4c-4673-468c-a40c-f01392dffc9b.client4659/status') 2024-09-15T23:36:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:51.529 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.4659 (3)192.168.0.1:0/2346903275 2024-09-15T23:36:51.529 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-09-15T23:36:51.530 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-09-15T23:36:51.530 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-09-15T23:36:51.530 INFO:teuthology.orchestra.run.smithi007.stderr:69 bytes copied, 2.4028e-05 s, 2.9 MB/s 2024-09-15T23:36:51.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:51.537 INFO:tasks.kclient:client.2 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v1'} 2024-09-15T23:36:51.537 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T23:36:51.537 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-15T23:36:51.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-15T23:36:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.640+0000 7fd0c42d2640 1 Processor -- start 2024-09-15T23:36:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.640+0000 7fd0c42d2640 1 -- start start 2024-09-15T23:36:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c42d2640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0bc15c530 0x7fd0bc0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c42d2640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd0bc0af150 0x7fd0bc09b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c42d2640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0bc09aa40 con 0x7fd0bc09b740 2024-09-15T23:36:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c42d2640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd0bc15c910 con 0x7fd0bc15c530 2024-09-15T23:36:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c42d2640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fd0bc0d0a20 con 0x7fd0bc0af150 2024-09-15T23:36:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c206e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0bc15c530 0x7fd0bc0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c286f640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0bc09b740 0x7fd0bc0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57764/0 (socket says 172.21.15.7:57764) 2024-09-15T23:36:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c286f640 1 -- 172.21.15.7:0/3530342899 learned_addr learned my addr 172.21.15.7:0/3530342899 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c186d640 1 --2- 172.21.15.7:0/3530342899 >> v2:172.21.15.18:3300/0 conn(0x7fd0bc0af150 0x7fd0bc09b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c206e640 1 -- 172.21.15.7:0/3530342899 >> v2:172.21.15.18:3300/0 conn(0x7fd0bc0af150 msgr2=0x7fd0bc09b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c206e640 1 --2- 172.21.15.7:0/3530342899 >> v2:172.21.15.18:3300/0 conn(0x7fd0bc0af150 0x7fd0bc09b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c206e640 1 -- 172.21.15.7:0/3530342899 >> v1:172.21.15.7:6789/0 conn(0x7fd0bc09b740 legacy=0x7fd0bc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c206e640 1 -- 172.21.15.7:0/3530342899 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc0d0c70 con 0x7fd0bc15c530 2024-09-15T23:36:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.641+0000 7fd0c106c640 1 -- 172.21.15.7:0/3530342899 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142666496 0 0) 0x7fd0bc09aa40 con 0x7fd0bc09b740 2024-09-15T23:36:51.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.642+0000 7fd0c206e640 1 --2- 172.21.15.7:0/3530342899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 0x7fd0bc0b1410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0c5ac0 tx=0x7fd0b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875d4cebdf57985 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:51.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.642+0000 7fd0c106c640 1 -- 172.21.15.7:0/3530342899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0037070 con 0x7fd0bc15c530 2024-09-15T23:36:51.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.642+0000 7fd0c106c640 1 -- 172.21.15.7:0/3530342899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b0032440 con 0x7fd0bc15c530 2024-09-15T23:36:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.642+0000 7fd0c106c640 1 -- 172.21.15.7:0/3530342899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0032810 con 0x7fd0bc15c530 2024-09-15T23:36:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.643+0000 7fd0c42d2640 1 -- 172.21.15.7:0/3530342899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 msgr2=0x7fd0bc0b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.643+0000 7fd0c42d2640 1 --2- 172.21.15.7:0/3530342899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 0x7fd0bc0b1410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0c5ac0 tx=0x7fd0b002a230 comp rx=0 tx=0).stop 2024-09-15T23:36:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.643+0000 7fd0c42d2640 1 -- 172.21.15.7:0/3530342899 shutdown_connections 2024-09-15T23:36:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.643+0000 7fd0c42d2640 1 --2- 172.21.15.7:0/3530342899 >> v2:172.21.15.18:3300/0 conn(0x7fd0bc0af150 0x7fd0bc09b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.643+0000 7fd0c42d2640 1 --2- 172.21.15.7:0/3530342899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 0x7fd0bc0b1410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.643+0000 7fd0c42d2640 1 -- 172.21.15.7:0/3530342899 >> 172.21.15.7:0/3530342899 conn(0x7fd0bc0b3f30 msgr2=0x7fd0bc0b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.643+0000 7fd0c42d2640 1 -- 172.21.15.7:0/3530342899 shutdown_connections 2024-09-15T23:36:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.643+0000 7fd0c42d2640 1 -- 172.21.15.7:0/3530342899 wait complete. 2024-09-15T23:36:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.644+0000 7fd0c42d2640 1 Processor -- start 2024-09-15T23:36:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.644+0000 7fd0c42d2640 1 -- start start 2024-09-15T23:36:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c42d2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0bc09b740 0x7fd0bc15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c42d2640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0bc0af150 0x7fd0bc0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c42d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 0x7fd0bc15a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c42d2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc0d1230 con 0x7fd0bc15c530 2024-09-15T23:36:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c42d2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc09ad70 con 0x7fd0bc09b740 2024-09-15T23:36:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c42d2640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fd0bc0af5c0 con 0x7fd0bc0af150 2024-09-15T23:36:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c286f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 0x7fd0bc15a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c286f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 0x7fd0bc15a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53850/0 (socket says 172.21.15.7:53850) 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c286f640 1 -- 172.21.15.7:0/743424710 learned_addr learned my addr 172.21.15.7:0/743424710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c206e640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0bc09b740 0x7fd0bc15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c186d640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0bc0af150 0x7fd0bc0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c286f640 1 -- 172.21.15.7:0/743424710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0bc0af150 msgr2=0x7fd0bc0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c286f640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0bc0af150 0x7fd0bc0c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c286f640 1 -- 172.21.15.7:0/743424710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0bc09b740 msgr2=0x7fd0bc15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c286f640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0bc09b740 0x7fd0bc15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.645+0000 7fd0c286f640 1 -- 172.21.15.7:0/743424710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc0b5ef0 con 0x7fd0bc15c530 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.646+0000 7fd0c286f640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 0x7fd0bc15a7c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c7a0 tx=0x7fd0b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.646+0000 7fd0aeffd640 1 -- 172.21.15.7:0/743424710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4018070 con 0x7fd0bc15c530 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.646+0000 7fd0aeffd640 1 -- 172.21.15.7:0/743424710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b400a640 con 0x7fd0bc15c530 2024-09-15T23:36:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.646+0000 7fd0aeffd640 1 -- 172.21.15.7:0/743424710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400aa70 con 0x7fd0bc15c530 2024-09-15T23:36:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.646+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0bc0cc020 con 0x7fd0bc15c530 2024-09-15T23:36:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.646+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0bc15ad00 con 0x7fd0bc15c530 2024-09-15T23:36:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.647+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0bc09bb90 con 0x7fd0bc15c530 2024-09-15T23:36:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.647+0000 7fd0aeffd640 1 -- 172.21.15.7:0/743424710 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd0b401b070 con 0x7fd0bc15c530 2024-09-15T23:36:51.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.648+0000 7fd0aeffd640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd0a006c2b0 0x7fd0a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:51.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.648+0000 7fd0aeffd640 1 -- 172.21.15.7:0/743424710 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd0b408ad20 con 0x7fd0bc15c530 2024-09-15T23:36:51.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.648+0000 7fd0c206e640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd0a006c2b0 0x7fd0a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:51.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.648+0000 7fd0c206e640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd0a006c2b0 0x7fd0a006e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0034000 tx=0x7fd0b002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:51.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.652+0000 7fd0aeffd640 1 -- 172.21.15.7:0/743424710 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd0b4058b00 con 0x7fd0bc15c530 2024-09-15T23:36:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.863+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd0bc0b0060 con 0x7fd0bc15c530 2024-09-15T23:36:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.863+0000 7fd0aeffd640 1 -- 172.21.15.7:0/743424710 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fd0b405ce00 con 0x7fd0bc15c530 2024-09-15T23:36:51.865 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T23:36:51.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd0a006c2b0 msgr2=0x7fd0a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:51.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd0a006c2b0 0x7fd0a006e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0034000 tx=0x7fd0b002e000 comp rx=0 tx=0).stop 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 msgr2=0x7fd0bc15a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 0x7fd0bc15a7c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c7a0 tx=0x7fd0b400cc60 comp rx=0 tx=0).stop 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 shutdown_connections 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0bc15c530 0x7fd0bc15a7c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd0a006c2b0 0x7fd0a006e770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0bc0af150 0x7fd0bc0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 --2- 172.21.15.7:0/743424710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0bc09b740 0x7fd0bc15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 >> 172.21.15.7:0/743424710 conn(0x7fd0bc0b3f30 msgr2=0x7fd0bc0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 shutdown_connections 2024-09-15T23:36:51.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:51.865+0000 7fd0c42d2640 1 -- 172.21.15.7:0/743424710 wait complete. 2024-09-15T23:36:51.878 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-15T23:36:51.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T23:36:51.878 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-15T23:36:51.878 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-15T23:36:51.878 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-15T23:36:51.878 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-15T23:36:51.879 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T23:36:51.879 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T23:36:51.879 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-15T23:36:51.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:51.879 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-15T23:36:51.895 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:36:51.895 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:36:51.895 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:36:51.895 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:51.895 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42744sec preferred_lft 42744sec 2024-09-15T23:36:51.896 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-15T23:36:51.897 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:51.897 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:51.897 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:51.897 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-15T23:36:51.897 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-15T23:36:51.897 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 5a:f1:a0:f2:08:e7 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:51.897 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-15T23:36:51.897 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:51.897 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::a0db:6aff:fe42:a0c6/64 scope link 2024-09-15T23:36:51.898 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:51.898 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-15T23:36:51.898 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 5a:f1:a0:f2:08:e7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T23:36:51.898 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::58f1:a0ff:fef2:8e7/64 scope link tentative 2024-09-15T23:36:51.898 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:51.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:51.898 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T23:36:51.961 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-15T23:36:51.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:51.962 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-15T23:36:52.019 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-15T23:36:52.019 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T23:36:52.020 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T23:36:52.020 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T23:36:52.020 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T23:36:52.020 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:52.020 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:52.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T23:36:52.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T23:36:52.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T23:36:52.020 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:52.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:52.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:52.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:52.148 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-15T23:36:52.164 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-15T23:36:52.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:52.164 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-09-15T23:36:52.219 INFO:teuthology.orchestra.run.smithi007.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-15T23:36:52.220 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:52.220 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:52.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-15T23:36:52.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-15T23:36:52.220 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:52.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:52.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:52.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-15T23:36:52.363 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 5e:90:19:8c:35:b1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::5c90:19ff:fe8c:35b1/64 scope link 2024-09-15T23:36:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:52.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:52.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-15T23:36:52.431 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:36:52.431 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:36:52.431 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:36:52.431 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:52.431 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-15T23:36:52.431 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:52.431 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-15T23:36:52.431 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 5e:90:19:8c:35:b1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-15T23:36:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-15T23:36:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::5c90:19ff:fe8c:35b1/64 scope link 2024-09-15T23:36:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:52.433 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-15T23:36:52.433 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:52.433 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:52.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-15T23:36:52.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-15T23:36:52.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-15T23:36:52.434 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-15T23:36:52.434 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-15T23:36:52.434 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:52.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:52.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:52.589 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:52.589 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:36:52.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.1 up 2024-09-15T23:36:52.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.1 master ceph-brx 2024-09-15T23:36:52.590 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:36:52.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:52.669 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:52.669 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-15T23:36:52.697 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-15T23:36:52.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:52.697 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-15T23:36:52.753 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-15T23:36:52.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:52.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs 2024-09-15T23:36:52.838 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs 2024-09-15T23:36:52.838 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs". 2024-09-15T23:36:52.838 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-15T23:36:52.847 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs 2024-09-15T23:36:52.847 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs". 2024-09-15T23:36:52.847 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-15T23:36:52.847 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:36:52.847 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:36:52.852 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs 2024-09-15T23:36:52.852 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs". 2024-09-15T23:36:52.852 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-15T23:36:52.853 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:36:52.853 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:36:52.853 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790" 2024-09-15T23:36:52.853 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790:/ 2024-09-15T23:36:52.853 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs,key=2,fsid=9da25f4c-4673-468c-a40c-f01392dffc9b" will pass to kernel 2024-09-15T23:36:52.855 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-15T23:36:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-15T23:36:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-15T23:36:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-15T23:36:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-15T23:36:52.856 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T23:36:52.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:52.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-15T23:36:52.886 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:52.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:52.985 INFO:teuthology.orchestra.run.smithi007.stdout:client4469 2024-09-15T23:36:52.985 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:36:52.985 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:36:53.002 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:53.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:53.101 INFO:teuthology.orchestra.run.smithi007.stdout:client4469 2024-09-15T23:36:53.102 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:53.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:53.158 INFO:teuthology.orchestra.run.smithi007.stdout:client4469 2024-09-15T23:36:53.158 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9da25f4c-4673-468c-a40c-f01392dffc9b.client4469/status') 2024-09-15T23:36:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:53.218 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.4469 (3)172.21.15.7:0/3765053312 2024-09-15T23:36:53.218 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-09-15T23:36:53.218 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-09-15T23:36:53.218 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-09-15T23:36:53.218 INFO:teuthology.orchestra.run.smithi007.stderr:69 bytes copied, 2.3407e-05 s, 2.9 MB/s 2024-09-15T23:36:53.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:53.225 INFO:tasks.kclient:client.1 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v1'} 2024-09-15T23:36:53.225 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T23:36:53.225 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-15T23:36:53.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-15T23:36:53.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.326+0000 7fc342b9f640 1 Processor -- start 2024-09-15T23:36:53.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.326+0000 7fc342b9f640 1 -- start start 2024-09-15T23:36:53.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc342b9f640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc33c0d51c0 0x7fc33c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc342b9f640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc33c0d30d0 0x7fc33c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc342b9f640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc33c0b3d10 con 0x7fc33c15e6d0 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc342b9f640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc33c0d28c0 con 0x7fc33c0d51c0 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc342b9f640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fc33c0d3500 con 0x7fc33c0d30d0 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc333fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc33c0d51c0 0x7fc33c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc333fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc33c0d51c0 0x7fc33c0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53860/0 (socket says 172.21.15.7:53860) 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc333fff640 1 -- 172.21.15.7:0/3508138437 learned_addr learned my addr 172.21.15.7:0/3508138437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc34113c640 1 --2- 172.21.15.7:0/3508138437 >> v2:172.21.15.18:3300/0 conn(0x7fc33c0d30d0 0x7fc33c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc333fff640 1 -- 172.21.15.7:0/3508138437 >> v2:172.21.15.18:3300/0 conn(0x7fc33c0d30d0 msgr2=0x7fc33c0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc333fff640 1 --2- 172.21.15.7:0/3508138437 >> v2:172.21.15.18:3300/0 conn(0x7fc33c0d30d0 0x7fc33c0b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc333fff640 1 -- 172.21.15.7:0/3508138437 >> v1:172.21.15.18:6789/0 conn(0x7fc33c15e6d0 legacy=0x7fc33c0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc333fff640 1 -- 172.21.15.7:0/3508138437 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33c0cfcd0 con 0x7fc33c0d51c0 2024-09-15T23:36:53.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.327+0000 7fc34113c640 1 --2- 172.21.15.7:0/3508138437 >> v2:172.21.15.18:3300/0 conn(0x7fc33c0d30d0 0x7fc33c0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:53.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.328+0000 7fc333fff640 1 --2- 172.21.15.7:0/3508138437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 0x7fc33c0d2b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0d3800 tx=0x7fc32c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b22a62eb30e847c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:53.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.328+0000 7fc3337fe640 1 -- 172.21.15.7:0/3508138437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c004030 con 0x7fc33c0d51c0 2024-09-15T23:36:53.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.328+0000 7fc3337fe640 1 -- 172.21.15.7:0/3508138437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc32c0041d0 con 0x7fc33c0d51c0 2024-09-15T23:36:53.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.328+0000 7fc3337fe640 1 -- 172.21.15.7:0/3508138437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c003610 con 0x7fc33c0d51c0 2024-09-15T23:36:53.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.328+0000 7fc342b9f640 1 -- 172.21.15.7:0/3508138437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 msgr2=0x7fc33c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:53.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.328+0000 7fc342b9f640 1 --2- 172.21.15.7:0/3508138437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 0x7fc33c0d2b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0d3800 tx=0x7fc32c02a230 comp rx=0 tx=0).stop 2024-09-15T23:36:53.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.329+0000 7fc342b9f640 1 -- 172.21.15.7:0/3508138437 shutdown_connections 2024-09-15T23:36:53.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.329+0000 7fc342b9f640 1 --2- 172.21.15.7:0/3508138437 >> v2:172.21.15.18:3300/0 conn(0x7fc33c0d30d0 0x7fc33c0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.329+0000 7fc342b9f640 1 --2- 172.21.15.7:0/3508138437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 0x7fc33c0d2b90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.329+0000 7fc342b9f640 1 -- 172.21.15.7:0/3508138437 >> 172.21.15.7:0/3508138437 conn(0x7fc33c08bfd0 msgr2=0x7fc33c0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:53.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.329+0000 7fc342b9f640 1 -- 172.21.15.7:0/3508138437 shutdown_connections 2024-09-15T23:36:53.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.329+0000 7fc342b9f640 1 -- 172.21.15.7:0/3508138437 wait complete. 2024-09-15T23:36:53.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.330+0000 7fc342b9f640 1 Processor -- start 2024-09-15T23:36:53.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.330+0000 7fc342b9f640 1 -- start start 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc342b9f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc33c0d30d0 0x7fc33c154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc342b9f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 0x7fc33c0ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc342b9f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc33c15e6d0 0x7fc33c0b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc342b9f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c15eab0 con 0x7fc33c0d51c0 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc342b9f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c0d0230 con 0x7fc33c15e6d0 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc342b9f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fc33c0d0510 con 0x7fc33c0d30d0 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc333fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 0x7fc33c0ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc333fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 0x7fc33c0ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53864/0 (socket says 172.21.15.7:53864) 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc333fff640 1 -- 172.21.15.7:0/1417522703 learned_addr learned my addr 172.21.15.7:0/1417522703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc34113c640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc33c15e6d0 0x7fc33c0b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc34093b640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc33c0d30d0 0x7fc33c154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:53.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc333fff640 1 -- 172.21.15.7:0/1417522703 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc33c0d30d0 msgr2=0x7fc33c154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc333fff640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc33c0d30d0 0x7fc33c154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc333fff640 1 -- 172.21.15.7:0/1417522703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc33c15e6d0 msgr2=0x7fc33c0b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc333fff640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc33c15e6d0 0x7fc33c0b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc333fff640 1 -- 172.21.15.7:0/1417522703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33c0c9b90 con 0x7fc33c0d51c0 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc34093b640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc33c0d30d0 0x7fc33c154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc333fff640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 0x7fc33c0ca950 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc32c034040 tx=0x7fc32c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.331+0000 7fc3317fa640 1 -- 172.21.15.7:0/1417522703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c041030 con 0x7fc33c0d51c0 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.332+0000 7fc3317fa640 1 -- 172.21.15.7:0/1417522703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc32c003c50 con 0x7fc33c0d51c0 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.332+0000 7fc3317fa640 1 -- 172.21.15.7:0/1417522703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c004030 con 0x7fc33c0d51c0 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.332+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33c0c9e70 con 0x7fc33c0d51c0 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.332+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33c0ca430 con 0x7fc33c0d51c0 2024-09-15T23:36:53.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.333+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33c0d2b90 con 0x7fc33c0d51c0 2024-09-15T23:36:53.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.333+0000 7fc3317fa640 1 -- 172.21.15.7:0/1417522703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc32c044070 con 0x7fc33c0d51c0 2024-09-15T23:36:53.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.334+0000 7fc3317fa640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc32806c2b0 0x7fc32806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:53.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.334+0000 7fc3317fa640 1 -- 172.21.15.7:0/1417522703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc32c0ac230 con 0x7fc33c0d51c0 2024-09-15T23:36:53.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.334+0000 7fc34093b640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc32806c2b0 0x7fc32806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:53.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.334+0000 7fc34093b640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc32806c2b0 0x7fc32806e770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc338006fd0 tx=0x7fc338008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:53.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.338+0000 7fc3317fa640 1 -- 172.21.15.7:0/1417522703 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc33c0cfcd0 con 0x7fc33c0d51c0 2024-09-15T23:36:53.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.553+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc33c0b3070 con 0x7fc33c0d51c0 2024-09-15T23:36:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.554+0000 7fc3317fa640 1 -- 172.21.15.7:0/1417522703 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fc32c003e20 con 0x7fc33c0d51c0 2024-09-15T23:36:53.556 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T23:36:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.555+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc32806c2b0 msgr2=0x7fc32806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc32806c2b0 0x7fc32806e770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc338006fd0 tx=0x7fc338008040 comp rx=0 tx=0).stop 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 msgr2=0x7fc33c0ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 0x7fc33c0ca950 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc32c034040 tx=0x7fc32c0046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 shutdown_connections 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc33c15e6d0 0x7fc33c0b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc33c0d51c0 0x7fc33c0ca950 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc32806c2b0 0x7fc32806e770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 --2- 172.21.15.7:0/1417522703 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc33c0d30d0 0x7fc33c154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 >> 172.21.15.7:0/1417522703 conn(0x7fc33c08bfd0 msgr2=0x7fc33c06ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 shutdown_connections 2024-09-15T23:36:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:53.556+0000 7fc342b9f640 1 -- 172.21.15.7:0/1417522703 wait complete. 2024-09-15T23:36:53.568 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-15T23:36:53.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T23:36:53.569 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi018.front.sepia.ceph.com 2024-09-15T23:36:53.569 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-15T23:36:53.569 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-15T23:36:53.569 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-15T23:36:53.569 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T23:36:53.569 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T23:36:53.569 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-15T23:36:53.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:53.569 DEBUG:teuthology.orchestra.run.smithi018:> ip addr 2024-09-15T23:36:53.589 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:36:53.589 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:36:53.589 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:36:53.589 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f0 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f1 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f0 2024-09-15T23:36:53.590 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T23:36:53.591 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42739sec preferred_lft 42739sec 2024-09-15T23:36:53.591 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2024-09-15T23:36:53.591 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:53.591 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:53.591 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:53.591 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f1 2024-09-15T23:36:53.591 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T23:36:53.591 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:53.591 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-09-15T23:36:53.591 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add name ceph-brx type bridge 2024-09-15T23:36:53.592 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr flush dev ceph-brx 2024-09-15T23:36:53.592 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set ceph-brx up 2024-09-15T23:36:53.592 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T23:36:53.592 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-09-15T23:36:53.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:53.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:53.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:53.823 DEBUG:teuthology.orchestra.run.smithi018:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T23:36:53.849 INFO:teuthology.orchestra.run.smithi018.stdout:1 2024-09-15T23:36:53.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:53.851 DEBUG:teuthology.orchestra.run.smithi018:> route 2024-09-15T23:36:53.924 INFO:teuthology.orchestra.run.smithi018.stdout:Kernel IP routing table 2024-09-15T23:36:53.924 INFO:teuthology.orchestra.run.smithi018.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T23:36:53.924 INFO:teuthology.orchestra.run.smithi018.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T23:36:53.924 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T23:36:53.924 INFO:teuthology.orchestra.run.smithi018.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T23:36:53.924 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:53.925 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-09-15T23:36:53.925 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T23:36:53.925 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T23:36:53.925 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T23:36:53.925 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-09-15T23:36:54.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:54.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:54.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:54.230 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-09-15T23:36:54.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:54.247 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list-id 2024-09-15T23:36:54.304 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:54.305 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-09-15T23:36:54.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-15T23:36:54.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-15T23:36:54.305 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-09-15T23:36:54.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:54.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:54.426 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-15T23:36:54.426 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:54.426 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-09-15T23:36:54.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-15T23:36:54.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T23:36:54.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-15T23:36:54.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-15T23:36:54.427 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-15T23:36:54.427 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-09-15T23:36:54.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:54.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:54.581 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:54.581 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-09-15T23:36:54.581 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set brx.0 up 2024-09-15T23:36:54.581 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T23:36:54.581 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-09-15T23:36:54.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:54.663 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:54.663 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-15T23:36:54.679 INFO:teuthology.orchestra.run.smithi018.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-15T23:36:54.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:54.679 DEBUG:teuthology.orchestra.run.smithi018:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-15T23:36:54.736 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-15T23:36:54.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:54.736 DEBUG:teuthology.orchestra.run.smithi018:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs 2024-09-15T23:36:54.851 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs 2024-09-15T23:36:54.851 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs". 2024-09-15T23:36:54.851 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-09-15T23:36:54.976 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs 2024-09-15T23:36:54.976 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs". 2024-09-15T23:36:54.976 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-09-15T23:36:54.976 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:36:54.976 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:36:54.981 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs 2024-09-15T23:36:54.981 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs". 2024-09-15T23:36:54.981 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-09-15T23:36:54.981 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:36:54.981 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:36:54.981 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790" 2024-09-15T23:36:54.982 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790:/ 2024-09-15T23:36:54.982 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs,key=1,fsid=9da25f4c-4673-468c-a40c-f01392dffc9b" will pass to kernel 2024-09-15T23:36:54.982 INFO:teuthology.orchestra.run.smithi018.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-15T23:36:54.982 INFO:teuthology.orchestra.run.smithi018.stdout: You just mounted a file system that supports labels which does not 2024-09-15T23:36:54.983 INFO:teuthology.orchestra.run.smithi018.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-15T23:36:54.983 INFO:teuthology.orchestra.run.smithi018.stdout: applications will generate AVC messages and not be allowed access to 2024-09-15T23:36:54.983 INFO:teuthology.orchestra.run.smithi018.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-15T23:36:54.985 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T23:36:54.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:54.985 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-15T23:36:55.011 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:55.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:55.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:55.109 INFO:teuthology.orchestra.run.smithi018.stdout:client4627 2024-09-15T23:36:55.110 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:36:55.110 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:36:55.127 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:55.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:55.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:55.223 INFO:teuthology.orchestra.run.smithi018.stdout:client4627 2024-09-15T23:36:55.224 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:55.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:55.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:55.281 INFO:teuthology.orchestra.run.smithi018.stdout:client4627 2024-09-15T23:36:55.281 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9da25f4c-4673-468c-a40c-f01392dffc9b.client4627/status') 2024-09-15T23:36:55.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:55.342 INFO:teuthology.orchestra.run.smithi018.stdout:instance: client.4627 (3)192.168.0.1:0/3984067396 2024-09-15T23:36:55.342 INFO:teuthology.orchestra.run.smithi018.stdout:blocklisted: false 2024-09-15T23:36:55.342 INFO:teuthology.orchestra.run.smithi018.stderr:0+1 records in 2024-09-15T23:36:55.342 INFO:teuthology.orchestra.run.smithi018.stderr:0+1 records out 2024-09-15T23:36:55.342 INFO:teuthology.orchestra.run.smithi018.stderr:69 bytes copied, 2.3567e-05 s, 2.9 MB/s 2024-09-15T23:36:55.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:55.349 INFO:tasks.kclient:client.3 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v1'} 2024-09-15T23:36:55.349 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T23:36:55.349 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-15T23:36:55.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-15T23:36:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.463+0000 7fda8ff1f640 1 Processor -- start 2024-09-15T23:36:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.463+0000 7fda8ff1f640 1 -- start start 2024-09-15T23:36:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8ff1f640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda88158e60 0x7fda880b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8ff1f640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fda880b31c0 0x7fda880d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8ff1f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda880b3bc0 0x7fda880d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8ff1f640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fda8806dfa0 con 0x7fda88158e60 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8ff1f640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fda88159240 con 0x7fda880b31c0 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8ff1f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fda8806e210 con 0x7fda880b3bc0 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8dcbb640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda88158e60 0x7fda880b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8dcbb640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda88158e60 0x7fda880b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53870/0 (socket says 172.21.15.7:53870) 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8dcbb640 1 -- 172.21.15.7:0/2215460180 learned_addr learned my addr 172.21.15.7:0/2215460180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8d4ba640 1 --2- 172.21.15.7:0/2215460180 >> v2:172.21.15.18:3300/0 conn(0x7fda880b31c0 0x7fda880d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8e4bc640 1 --2- 172.21.15.7:0/2215460180 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda880b3bc0 0x7fda880d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8dcbb640 1 -- 172.21.15.7:0/2215460180 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda880b3bc0 msgr2=0x7fda880d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8dcbb640 1 --2- 172.21.15.7:0/2215460180 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda880b3bc0 0x7fda880d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8dcbb640 1 -- 172.21.15.7:0/2215460180 >> v2:172.21.15.18:3300/0 conn(0x7fda880b31c0 msgr2=0x7fda880d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8dcbb640 1 --2- 172.21.15.7:0/2215460180 >> v2:172.21.15.18:3300/0 conn(0x7fda880b31c0 0x7fda880d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.464+0000 7fda8dcbb640 1 -- 172.21.15.7:0/2215460180 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8806e4f0 con 0x7fda88158e60 2024-09-15T23:36:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.465+0000 7fda8dcbb640 1 --2- 172.21.15.7:0/2215460180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda88158e60 0x7fda880b3680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fda8815b860 tx=0x7fda8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a04f6b08fb703c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.465+0000 7fda8ccb9640 1 -- 172.21.15.7:0/2215460180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84037070 con 0x7fda88158e60 2024-09-15T23:36:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.465+0000 7fda8ccb9640 1 -- 172.21.15.7:0/2215460180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda84032440 con 0x7fda88158e60 2024-09-15T23:36:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.465+0000 7fda8ccb9640 1 -- 172.21.15.7:0/2215460180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84032810 con 0x7fda88158e60 2024-09-15T23:36:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.466+0000 7fda8ff1f640 1 -- 172.21.15.7:0/2215460180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda88158e60 msgr2=0x7fda880b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.466+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/2215460180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda88158e60 0x7fda880b3680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fda8815b860 tx=0x7fda8402a270 comp rx=0 tx=0).stop 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.466+0000 7fda8ff1f640 1 -- 172.21.15.7:0/2215460180 shutdown_connections 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.466+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/2215460180 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda880b3bc0 0x7fda880d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.466+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/2215460180 >> v2:172.21.15.18:3300/0 conn(0x7fda880b31c0 0x7fda880d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.466+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/2215460180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda88158e60 0x7fda880b3680 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.466+0000 7fda8ff1f640 1 -- 172.21.15.7:0/2215460180 >> 172.21.15.7:0/2215460180 conn(0x7fda880b21e0 msgr2=0x7fda880b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.466+0000 7fda8ff1f640 1 -- 172.21.15.7:0/2215460180 shutdown_connections 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.466+0000 7fda8ff1f640 1 -- 172.21.15.7:0/2215460180 wait complete. 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.467+0000 7fda8ff1f640 1 Processor -- start 2024-09-15T23:36:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.467+0000 7fda8ff1f640 1 -- start start 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8ff1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda880b31c0 0x7fda88148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8ff1f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda880b3bc0 0x7fda880af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8ff1f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda88158e60 0x7fda8809aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8ff1f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda8815ca20 con 0x7fda880b31c0 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8ff1f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fda8806eab0 con 0x7fda880b3bc0 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8ff1f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fda8815cba0 con 0x7fda88158e60 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8dcbb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda880b31c0 0x7fda88148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8dcbb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda880b31c0 0x7fda88148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53884/0 (socket says 172.21.15.7:53884) 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8dcbb640 1 -- 172.21.15.7:0/1257425510 learned_addr learned my addr 172.21.15.7:0/1257425510 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8e4bc640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda88158e60 0x7fda8809aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8d4ba640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda880b3bc0 0x7fda880af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8dcbb640 1 -- 172.21.15.7:0/1257425510 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda88158e60 msgr2=0x7fda8809aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8dcbb640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda88158e60 0x7fda8809aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8dcbb640 1 -- 172.21.15.7:0/1257425510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda880b3bc0 msgr2=0x7fda880af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8dcbb640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda880b3bc0 0x7fda880af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.468+0000 7fda8dcbb640 1 -- 172.21.15.7:0/1257425510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda88148aa0 con 0x7fda880b31c0 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.469+0000 7fda8dcbb640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda880b31c0 0x7fda88148560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fda8402a2b0 tx=0x7fda840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.469+0000 7fda7e7fc640 1 -- 172.21.15.7:0/1257425510 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84041020 con 0x7fda880b31c0 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.469+0000 7fda7e7fc640 1 -- 172.21.15.7:0/1257425510 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda8403d620 con 0x7fda880b31c0 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.469+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda880af990 con 0x7fda880b31c0 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.469+0000 7fda7e7fc640 1 -- 172.21.15.7:0/1257425510 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84037040 con 0x7fda880b31c0 2024-09-15T23:36:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.469+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8809a2e0 con 0x7fda880b31c0 2024-09-15T23:36:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.470+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda880b3680 con 0x7fda880b31c0 2024-09-15T23:36:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.470+0000 7fda7e7fc640 1 -- 172.21.15.7:0/1257425510 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fda84044020 con 0x7fda880b31c0 2024-09-15T23:36:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.471+0000 7fda7e7fc640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda6c06c2b0 0x7fda6c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.471+0000 7fda7e7fc640 1 -- 172.21.15.7:0/1257425510 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fda840abc60 con 0x7fda880b31c0 2024-09-15T23:36:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.471+0000 7fda8d4ba640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda6c06c2b0 0x7fda6c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.471+0000 7fda8d4ba640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda6c06c2b0 0x7fda6c06e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fda8806fea0 tx=0x7fda78009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:55.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.475+0000 7fda7e7fc640 1 -- 172.21.15.7:0/1257425510 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fda8404b030 con 0x7fda880b31c0 2024-09-15T23:36:55.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:36:55.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:36:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.691+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fda880afeb0 con 0x7fda880b31c0 2024-09-15T23:36:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.691+0000 7fda7e7fc640 1 -- 172.21.15.7:0/1257425510 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fda84079ad0 con 0x7fda880b31c0 2024-09-15T23:36:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T23:36:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda6c06c2b0 msgr2=0x7fda6c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda6c06c2b0 0x7fda6c06e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fda8806fea0 tx=0x7fda78009190 comp rx=0 tx=0).stop 2024-09-15T23:36:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda880b31c0 msgr2=0x7fda88148560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda880b31c0 0x7fda88148560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fda8402a2b0 tx=0x7fda840046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 shutdown_connections 2024-09-15T23:36:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda6c06c2b0 0x7fda6c06e770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda88158e60 0x7fda8809aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda880b3bc0 0x7fda880af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 --2- 172.21.15.7:0/1257425510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda880b31c0 0x7fda88148560 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 >> 172.21.15.7:0/1257425510 conn(0x7fda880b21e0 msgr2=0x7fda8815c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 shutdown_connections 2024-09-15T23:36:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:55.693+0000 7fda8ff1f640 1 -- 172.21.15.7:0/1257425510 wait complete. 2024-09-15T23:36:55.706 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-15T23:36:55.706 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T23:36:55.706 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi018.front.sepia.ceph.com 2024-09-15T23:36:55.706 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-15T23:36:55.706 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-15T23:36:55.706 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-15T23:36:55.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T23:36:55.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T23:36:55.707 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-15T23:36:55.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:55.707 DEBUG:teuthology.orchestra.run.smithi018:> ip addr 2024-09-15T23:36:55.724 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:36:55.724 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:36:55.724 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:36:55.724 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f0 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f1 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f0 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T23:36:55.725 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f1 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether ee:e8:57:5e:52:a4 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::7c2f:dcff:feec:d0b6/64 scope link 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-15T23:36:55.726 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether ee:e8:57:5e:52:a4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-15T23:36:55.727 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ece8:57ff:fe5e:52a4/64 scope link tentative 2024-09-15T23:36:55.727 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:55.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:55.727 DEBUG:teuthology.orchestra.run.smithi018:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T23:36:55.789 INFO:teuthology.orchestra.run.smithi018.stdout:1 2024-09-15T23:36:55.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:55.791 DEBUG:teuthology.orchestra.run.smithi018:> route 2024-09-15T23:36:55.848 INFO:teuthology.orchestra.run.smithi018.stdout:Kernel IP routing table 2024-09-15T23:36:55.848 INFO:teuthology.orchestra.run.smithi018.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T23:36:55.849 INFO:teuthology.orchestra.run.smithi018.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T23:36:55.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T23:36:55.849 INFO:teuthology.orchestra.run.smithi018.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T23:36:55.849 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:55.850 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-09-15T23:36:55.850 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T23:36:55.850 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T23:36:55.850 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T23:36:55.850 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-09-15T23:36:55.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:55.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:55.979 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-09-15T23:36:55.995 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-15T23:36:55.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:55.996 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list-id 2024-09-15T23:36:56.052 INFO:teuthology.orchestra.run.smithi018.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-15T23:36:56.052 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:56.052 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-09-15T23:36:56.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-15T23:36:56.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-15T23:36:56.053 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-09-15T23:36:56.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:56.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:56.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:56.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-15T23:36:56.196 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:36:56.196 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 5a:65:11:a1:bf:0b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::5865:11ff:fea1:bf0b/64 scope link 2024-09-15T23:36:56.197 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:56.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:56.198 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-15T23:36:56.264 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:36:56.264 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 5a:65:11:a1:bf:0b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::5865:11ff:fea1:bf0b/64 scope link 2024-09-15T23:36:56.265 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:36:56.266 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-15T23:36:56.266 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:56.266 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-09-15T23:36:56.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-15T23:36:56.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-15T23:36:56.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-15T23:36:56.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-15T23:36:56.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-15T23:36:56.267 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-09-15T23:36:56.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:56.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:56.423 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:36:56.423 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-09-15T23:36:56.423 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set brx.1 up 2024-09-15T23:36:56.423 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set dev brx.1 master ceph-brx 2024-09-15T23:36:56.423 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-09-15T23:36:56.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:56.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:56.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:56.509 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-15T23:36:56.525 INFO:teuthology.orchestra.run.smithi018.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-15T23:36:56.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:36:56.525 DEBUG:teuthology.orchestra.run.smithi018:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-15T23:36:56.581 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-15T23:36:56.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:56.582 DEBUG:teuthology.orchestra.run.smithi018:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs 2024-09-15T23:36:56.667 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs 2024-09-15T23:36:56.667 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs". 2024-09-15T23:36:56.667 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-09-15T23:36:56.676 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs 2024-09-15T23:36:56.676 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs". 2024-09-15T23:36:56.676 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-09-15T23:36:56.676 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:36:56.676 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:36:56.682 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs 2024-09-15T23:36:56.682 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs". 2024-09-15T23:36:56.682 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-09-15T23:36:56.682 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:36:56.682 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:36:56.682 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790" 2024-09-15T23:36:56.682 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790:/ 2024-09-15T23:36:56.682 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs,key=3,fsid=9da25f4c-4673-468c-a40c-f01392dffc9b" will pass to kernel 2024-09-15T23:36:56.683 INFO:teuthology.orchestra.run.smithi018.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-15T23:36:56.683 INFO:teuthology.orchestra.run.smithi018.stdout: You just mounted a file system that supports labels which does not 2024-09-15T23:36:56.683 INFO:teuthology.orchestra.run.smithi018.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-15T23:36:56.683 INFO:teuthology.orchestra.run.smithi018.stdout: applications will generate AVC messages and not be allowed access to 2024-09-15T23:36:56.683 INFO:teuthology.orchestra.run.smithi018.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-15T23:36:56.684 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T23:36:56.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:36:56.685 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-15T23:36:56.710 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:56.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:56.807 INFO:teuthology.orchestra.run.smithi018.stdout:client4683 2024-09-15T23:36:56.807 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:36:56.807 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:36:56.824 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:56.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:56.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:56.922 INFO:teuthology.orchestra.run.smithi018.stdout:client4683 2024-09-15T23:36:56.922 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:36:56.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:36:56.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:56.979 INFO:teuthology.orchestra.run.smithi018.stdout:client4683 2024-09-15T23:36:56.979 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9da25f4c-4673-468c-a40c-f01392dffc9b.client4683/status') 2024-09-15T23:36:57.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:36:57.039 INFO:teuthology.orchestra.run.smithi018.stdout:instance: client.4683 (3)172.21.15.18:0/2650543846 2024-09-15T23:36:57.039 INFO:teuthology.orchestra.run.smithi018.stdout:blocklisted: false 2024-09-15T23:36:57.039 INFO:teuthology.orchestra.run.smithi018.stderr:0+1 records in 2024-09-15T23:36:57.039 INFO:teuthology.orchestra.run.smithi018.stderr:0+1 records out 2024-09-15T23:36:57.039 INFO:teuthology.orchestra.run.smithi018.stderr:70 bytes copied, 2.39e-05 s, 2.9 MB/s 2024-09-15T23:36:57.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-15T23:36:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:36:57.046 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-15T23:36:57.125 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2024-09-15T23:36:57.125 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2024-09-15T23:36:57.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:36:57.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.238+0000 7fda749b7640 1 Processor -- start 2024-09-15T23:36:57.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.239+0000 7fda749b7640 1 -- start start 2024-09-15T23:36:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.239+0000 7fda749b7640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda7015e840 0x7fda700d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.239+0000 7fda749b7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda700d0f90 0x7fda700d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda749b7640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda7006dd80 con 0x7fda700d52e0 2024-09-15T23:36:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda749b7640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fda700d1940 con 0x7fda7015e840 2024-09-15T23:36:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda749b7640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fda700cfe90 con 0x7fda700d0f90 2024-09-15T23:36:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda6e59c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda7015e840 0x7fda700d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda6e59c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda7015e840 0x7fda700d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60300/0 (socket says 172.21.15.7:60300) 2024-09-15T23:36:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda6e59c640 1 -- 172.21.15.7:0/3815520889 learned_addr learned my addr 172.21.15.7:0/3815520889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda6ed9d640 1 --2- 172.21.15.7:0/3815520889 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda700d0f90 0x7fda700d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda6e59c640 1 -- 172.21.15.7:0/3815520889 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda700d0f90 msgr2=0x7fda700d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda6e59c640 1 --2- 172.21.15.7:0/3815520889 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda700d0f90 0x7fda700d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda6e59c640 1 -- 172.21.15.7:0/3815520889 >> v1:172.21.15.18:6789/0 conn(0x7fda700d52e0 legacy=0x7fda7006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.240+0000 7fda6e59c640 1 -- 172.21.15.7:0/3815520889 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda700cf560 con 0x7fda7015e840 2024-09-15T23:36:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.241+0000 7fda6e59c640 1 --2- 172.21.15.7:0/3815520889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7015e840 0x7fda700d0320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda60009870 tx=0x7fda6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c7e68078be20a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.241+0000 7fda6d59a640 1 -- 172.21.15.7:0/3815520889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60037070 con 0x7fda7015e840 2024-09-15T23:36:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.241+0000 7fda6d59a640 1 -- 172.21.15.7:0/3815520889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda60032440 con 0x7fda7015e840 2024-09-15T23:36:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.241+0000 7fda6d59a640 1 -- 172.21.15.7:0/3815520889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60032810 con 0x7fda7015e840 2024-09-15T23:36:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.242+0000 7fda749b7640 1 -- 172.21.15.7:0/3815520889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7015e840 msgr2=0x7fda700d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.242+0000 7fda749b7640 1 --2- 172.21.15.7:0/3815520889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7015e840 0x7fda700d0320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda60009870 tx=0x7fda6002a270 comp rx=0 tx=0).stop 2024-09-15T23:36:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.242+0000 7fda749b7640 1 -- 172.21.15.7:0/3815520889 shutdown_connections 2024-09-15T23:36:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.242+0000 7fda749b7640 1 --2- 172.21.15.7:0/3815520889 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda700d0f90 0x7fda700d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.242+0000 7fda749b7640 1 --2- 172.21.15.7:0/3815520889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7015e840 0x7fda700d0320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.242+0000 7fda749b7640 1 -- 172.21.15.7:0/3815520889 >> 172.21.15.7:0/3815520889 conn(0x7fda7008bfd0 msgr2=0x7fda700d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:57.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.242+0000 7fda749b7640 1 -- 172.21.15.7:0/3815520889 shutdown_connections 2024-09-15T23:36:57.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.242+0000 7fda749b7640 1 -- 172.21.15.7:0/3815520889 wait complete. 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.243+0000 7fda749b7640 1 Processor -- start 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.243+0000 7fda749b7640 1 -- start start 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda749b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda700d0f90 0x7fda700b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda749b7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda700d52e0 0x7fda7015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda749b7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda7015e840 0x7fda701578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda749b7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda7006e030 con 0x7fda700d0f90 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda749b7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fda700cfb20 con 0x7fda700d52e0 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda749b7640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fda7015ec20 con 0x7fda7015e840 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda700d0f90 0x7fda700b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda700d0f90 0x7fda700b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60308/0 (socket says 172.21.15.7:60308) 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6e59c640 1 -- 172.21.15.7:0/2450477875 learned_addr learned my addr 172.21.15.7:0/2450477875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6ed9d640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda7015e840 0x7fda701578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6dd9b640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda700d52e0 0x7fda7015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6e59c640 1 -- 172.21.15.7:0/2450477875 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda7015e840 msgr2=0x7fda701578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6e59c640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda7015e840 0x7fda701578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6e59c640 1 -- 172.21.15.7:0/2450477875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda700d52e0 msgr2=0x7fda7015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6e59c640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda700d52e0 0x7fda7015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.244+0000 7fda6e59c640 1 -- 172.21.15.7:0/2450477875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda700b5020 con 0x7fda700d0f90 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.245+0000 7fda6e59c640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda700d0f90 0x7fda700b63d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fda6002a2b0 tx=0x7fda600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.245+0000 7fda5effd640 1 -- 172.21.15.7:0/2450477875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60041020 con 0x7fda700d0f90 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.245+0000 7fda5effd640 1 -- 172.21.15.7:0/2450477875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda600306c0 con 0x7fda700d0f90 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.245+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda700b52c0 con 0x7fda700d0f90 2024-09-15T23:36:57.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.245+0000 7fda5effd640 1 -- 172.21.15.7:0/2450477875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60037040 con 0x7fda700d0f90 2024-09-15T23:36:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.245+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda701588c0 con 0x7fda700d0f90 2024-09-15T23:36:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.246+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda701586d0 con 0x7fda700d0f90 2024-09-15T23:36:57.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.246+0000 7fda5effd640 1 -- 172.21.15.7:0/2450477875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fda60044020 con 0x7fda700d0f90 2024-09-15T23:36:57.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.247+0000 7fda5effd640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda5006c2b0 0x7fda5006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.247+0000 7fda5effd640 1 -- 172.21.15.7:0/2450477875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fda600abdb0 con 0x7fda700d0f90 2024-09-15T23:36:57.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.247+0000 7fda6dd9b640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda5006c2b0 0x7fda5006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.247+0000 7fda6dd9b640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda5006c2b0 0x7fda5006e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda700b8a00 tx=0x7fda64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:57.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.251+0000 7fda5effd640 1 -- 172.21.15.7:0/2450477875 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fda6004b030 con 0x7fda700d0f90 2024-09-15T23:36:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.429+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda700b8670 con 0x7fda700d0f90 2024-09-15T23:36:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.431+0000 7fda5effd640 1 -- 172.21.15.7:0/2450477875 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7fda60079c20 con 0x7fda700d0f90 2024-09-15T23:36:57.433 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:36:57.433 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:36:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda5006c2b0 msgr2=0x7fda5006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda5006c2b0 0x7fda5006e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda700b8a00 tx=0x7fda64009040 comp rx=0 tx=0).stop 2024-09-15T23:36:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda700d0f90 msgr2=0x7fda700b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda700d0f90 0x7fda700b63d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fda6002a2b0 tx=0x7fda600046c0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 shutdown_connections 2024-09-15T23:36:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fda5006c2b0 0x7fda5006e770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda7015e840 0x7fda701578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda700d52e0 0x7fda7015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 --2- 172.21.15.7:0/2450477875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda700d0f90 0x7fda700b63d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 >> 172.21.15.7:0/2450477875 conn(0x7fda7008bfd0 msgr2=0x7fda700d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 shutdown_connections 2024-09-15T23:36:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.433+0000 7fda749b7640 1 -- 172.21.15.7:0/2450477875 wait complete. 2024-09-15T23:36:57.446 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T23:36:09.860411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T23:36:20.781569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T23:36:21.898266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-15T23:36:25.611819+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-15T23:36:57.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T23:36:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.551+0000 7fad4edb7640 1 Processor -- start 2024-09-15T23:36:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.551+0000 7fad4edb7640 1 -- start start 2024-09-15T23:36:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.552+0000 7fad4edb7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad48070ea0 0x7fad480d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.552+0000 7fad4edb7640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad4809b450 0x7fad480b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.552+0000 7fad4edb7640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4809bae0 con 0x7fad4815e850 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.552+0000 7fad4edb7640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fad480d05b0 con 0x7fad4809b450 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.552+0000 7fad4edb7640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fad480d1110 con 0x7fad48070ea0 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.552+0000 7fad4d354640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad4809b450 0x7fad480b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.552+0000 7fad4d354640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad4809b450 0x7fad480b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60310/0 (socket says 172.21.15.7:60310) 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.552+0000 7fad4d354640 1 -- 172.21.15.7:0/237594281 learned_addr learned my addr 172.21.15.7:0/237594281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.552+0000 7fad47fff640 1 --2- 172.21.15.7:0/237594281 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad48070ea0 0x7fad480d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.553+0000 7fad4d354640 1 -- 172.21.15.7:0/237594281 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad48070ea0 msgr2=0x7fad480d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.553+0000 7fad4d354640 1 --2- 172.21.15.7:0/237594281 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad48070ea0 0x7fad480d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.553+0000 7fad4d354640 1 -- 172.21.15.7:0/237594281 >> v1:172.21.15.18:6789/0 conn(0x7fad4815e850 legacy=0x7fad48070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.553+0000 7fad4d354640 1 -- 172.21.15.7:0/237594281 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad480d07e0 con 0x7fad4809b450 2024-09-15T23:36:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.553+0000 7fad47fff640 1 --2- 172.21.15.7:0/237594281 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad48070ea0 0x7fad480d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:36:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.553+0000 7fad4d354640 1 --2- 172.21.15.7:0/237594281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad4809b450 0x7fad480b18c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fad40009e50 tx=0x7fad4002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a959c0e3e31f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.553+0000 7fad477fe640 1 -- 172.21.15.7:0/237594281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40036070 con 0x7fad4809b450 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.553+0000 7fad477fe640 1 -- 172.21.15.7:0/237594281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad40031600 con 0x7fad4809b450 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.553+0000 7fad477fe640 1 -- 172.21.15.7:0/237594281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad400319d0 con 0x7fad4809b450 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.554+0000 7fad4edb7640 1 -- 172.21.15.7:0/237594281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad4809b450 msgr2=0x7fad480b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.554+0000 7fad4edb7640 1 --2- 172.21.15.7:0/237594281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad4809b450 0x7fad480b18c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fad40009e50 tx=0x7fad4002a710 comp rx=0 tx=0).stop 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.554+0000 7fad4edb7640 1 -- 172.21.15.7:0/237594281 shutdown_connections 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.554+0000 7fad4edb7640 1 --2- 172.21.15.7:0/237594281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad4809b450 0x7fad480b18c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.554+0000 7fad4edb7640 1 --2- 172.21.15.7:0/237594281 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad48070ea0 0x7fad480d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.554+0000 7fad4edb7640 1 -- 172.21.15.7:0/237594281 >> 172.21.15.7:0/237594281 conn(0x7fad480d2420 msgr2=0x7fad480d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.554+0000 7fad4edb7640 1 -- 172.21.15.7:0/237594281 shutdown_connections 2024-09-15T23:36:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.554+0000 7fad4edb7640 1 -- 172.21.15.7:0/237594281 wait complete. 2024-09-15T23:36:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.555+0000 7fad4edb7640 1 Processor -- start 2024-09-15T23:36:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.555+0000 7fad4edb7640 1 -- start start 2024-09-15T23:36:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4edb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad48070ea0 0x7fad48160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4edb7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad4809b450 0x7fad48157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4edb7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad4815e850 0x7fad480c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4edb7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad4815ec30 con 0x7fad48070ea0 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4edb7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fad480d0da0 con 0x7fad4815e850 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4edb7640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fad4806cca0 con 0x7fad4809b450 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4cb53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad48070ea0 0x7fad48160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4cb53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad48070ea0 0x7fad48160a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60316/0 (socket says 172.21.15.7:60316) 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4cb53640 1 -- 172.21.15.7:0/1054622652 learned_addr learned my addr 172.21.15.7:0/1054622652 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad47fff640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad4809b450 0x7fad48157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.556+0000 7fad4d354640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad4815e850 0x7fad480c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad4cb53640 1 -- 172.21.15.7:0/1054622652 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad4809b450 msgr2=0x7fad48157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad4cb53640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad4809b450 0x7fad48157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad4cb53640 1 -- 172.21.15.7:0/1054622652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad4815e850 msgr2=0x7fad480c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad4cb53640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad4815e850 0x7fad480c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad4cb53640 1 -- 172.21.15.7:0/1054622652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad481615e0 con 0x7fad48070ea0 2024-09-15T23:36:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad4cb53640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad48070ea0 0x7fad48160a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c3f0 tx=0x7fad3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad457fa640 1 -- 172.21.15.7:0/1054622652 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad380052d0 con 0x7fad48070ea0 2024-09-15T23:36:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad457fa640 1 -- 172.21.15.7:0/1054622652 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad38005490 con 0x7fad48070ea0 2024-09-15T23:36:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad48161300 con 0x7fad48070ea0 2024-09-15T23:36:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad457fa640 1 -- 172.21.15.7:0/1054622652 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38007260 con 0x7fad48070ea0 2024-09-15T23:36:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.557+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad481f7fa0 con 0x7fad48070ea0 2024-09-15T23:36:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.558+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad480d15a0 con 0x7fad48070ea0 2024-09-15T23:36:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.559+0000 7fad457fa640 1 -- 172.21.15.7:0/1054622652 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fad38007400 con 0x7fad48070ea0 2024-09-15T23:36:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.559+0000 7fad457fa640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fad3406c2b0 0x7fad3406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.559+0000 7fad457fa640 1 -- 172.21.15.7:0/1054622652 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fad3808ab80 con 0x7fad48070ea0 2024-09-15T23:36:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.559+0000 7fad47fff640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fad3406c2b0 0x7fad3406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.560+0000 7fad47fff640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fad3406c2b0 0x7fad3406e770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fad481f7aa0 tx=0x7fad3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.564+0000 7fad457fa640 1 -- 172.21.15.7:0/1054622652 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad38058960 con 0x7fad48070ea0 2024-09-15T23:36:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.738+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad480b83a0 con 0x7fad48070ea0 2024-09-15T23:36:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.738+0000 7fad457fa640 1 -- 172.21.15.7:0/1054622652 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fad3805cc60 con 0x7fad48070ea0 2024-09-15T23:36:57.740 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-15T23:36:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fad3406c2b0 msgr2=0x7fad3406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fad3406c2b0 0x7fad3406e770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fad481f7aa0 tx=0x7fad3c008040 comp rx=0 tx=0).stop 2024-09-15T23:36:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad48070ea0 msgr2=0x7fad48160a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad48070ea0 0x7fad48160a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c3f0 tx=0x7fad3800c8b0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 shutdown_connections 2024-09-15T23:36:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad4815e850 0x7fad480c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fad3406c2b0 0x7fad3406e770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad4809b450 0x7fad48157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 --2- 172.21.15.7:0/1054622652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad48070ea0 0x7fad48160a70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 >> 172.21.15.7:0/1054622652 conn(0x7fad480d2420 msgr2=0x7fad480b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.740+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 shutdown_connections 2024-09-15T23:36:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.741+0000 7fad4edb7640 1 -- 172.21.15.7:0/1054622652 wait complete. 2024-09-15T23:36:57.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-15T23:36:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.858+0000 7f2375f18640 1 Processor -- start 2024-09-15T23:36:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.858+0000 7f2375f18640 1 -- start start 2024-09-15T23:36:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.858+0000 7f2375f18640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f237009b150 0x7f23700b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.858+0000 7f2375f18640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23700b4ea0 con 0x7f23700b4a60 2024-09-15T23:36:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.858+0000 7f2375f18640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f237009b5e0 con 0x7f237015ebd0 2024-09-15T23:36:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.858+0000 7f2375f18640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f23700b3640 con 0x7f237009b150 2024-09-15T23:36:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236effd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f23700b4a60 0x7f237006d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56764/0 (socket says 172.21.15.7:56764) 2024-09-15T23:36:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236effd640 1 -- 172.21.15.7:0/3284045582 learned_addr learned my addr 172.21.15.7:0/3284045582 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236ffff640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f237009b150 0x7f23700b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1759298065 0 0) 0x7f23700b4ea0 con 0x7f23700b4a60 2024-09-15T23:36:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2358003520 con 0x7f23700b4a60 2024-09-15T23:36:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 54435301 0 0) 0x7f2358003520 con 0x7f23700b4a60 2024-09-15T23:36:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f237009b150 msgr2=0x7f23700b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f237009b150 0x7f23700b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 >> v1:172.21.15.18:6789/0 conn(0x7f237015ebd0 legacy=0x7f237009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23700b2d00 con 0x7f23700b4a60 2024-09-15T23:36:57.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f2364001670 con 0x7f23700b4a60 2024-09-15T23:36:57.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 >> v1:172.21.15.7:6789/0 conn(0x7f23700b4a60 legacy=0x7f237006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23580039e0 0x7f2358003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2358004800 0x7f2358004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f237009b150 0x7f23580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23700b3640 con 0x7f23580039e0 2024-09-15T23:36:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f235800ba70 con 0x7f2358004800 2024-09-15T23:36:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f235800bc70 con 0x7f237009b150 2024-09-15T23:36:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236f7fe640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23580039e0 0x7f2358003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236effd640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2358004800 0x7f2358004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.859+0000 7f236ffff640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f237009b150 0x7f23580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236f7fe640 1 -- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f237009b150 msgr2=0x7f23580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236f7fe640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f237009b150 0x7f23580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236f7fe640 1 -- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2358004800 msgr2=0x7f2358004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236f7fe640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2358004800 0x7f2358004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236f7fe640 1 -- 172.21.15.7:0/3284045582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2358003520 con 0x7f23580039e0 2024-09-15T23:36:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236ffff640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f237009b150 0x7f23580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236f7fe640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23580039e0 0x7f2358003e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2360001720 tx=0x7f2358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f5c46fd4922794 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360001370 con 0x7f23580039e0 2024-09-15T23:36:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23600052d0 con 0x7f23580039e0 2024-09-15T23:36:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.860+0000 7f236e7fc640 1 -- 172.21.15.7:0/3284045582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236002ab90 con 0x7f23580039e0 2024-09-15T23:36:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.861+0000 7f2375f18640 1 -- 172.21.15.7:0/3284045582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23580039e0 msgr2=0x7f2358003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.861+0000 7f2375f18640 1 --2- 172.21.15.7:0/3284045582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23580039e0 0x7f2358003e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2360001720 tx=0x7f2358005830 comp rx=0 tx=0).stop 2024-09-15T23:36:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.861+0000 7f236ffff640 1 -- 172.21.15.7:0/3284045582 reap_dead start 2024-09-15T23:36:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.861+0000 7f2375f18640 1 -- 172.21.15.7:0/3284045582 shutdown_connections 2024-09-15T23:36:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.861+0000 7f2375f18640 1 -- 172.21.15.7:0/3284045582 >> 172.21.15.7:0/3284045582 conn(0x7f23700b3f40 msgr2=0x7f23700b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.861+0000 7f2375f18640 1 -- 172.21.15.7:0/3284045582 shutdown_connections 2024-09-15T23:36:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.861+0000 7f2375f18640 1 -- 172.21.15.7:0/3284045582 wait complete. 2024-09-15T23:36:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.862+0000 7f2375f18640 1 Processor -- start 2024-09-15T23:36:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.862+0000 7f2375f18640 1 -- start start 2024-09-15T23:36:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f2375f18640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23700b6b20 0x7f23701f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f2375f18640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23700b6f00 0x7f23700b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f2375f18640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700b2470 0x7f23701574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f2375f18640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f237009b850 con 0x7f23700b6f00 2024-09-15T23:36:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f2375f18640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f237009bbc0 con 0x7f23700b2470 2024-09-15T23:36:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f2375f18640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f237015efb0 con 0x7f23700b6b20 2024-09-15T23:36:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f236effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23700b6f00 0x7f23700b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f236effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23700b6f00 0x7f23700b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60340/0 (socket says 172.21.15.7:60340) 2024-09-15T23:36:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f236effd640 1 -- 172.21.15.7:0/2148772691 learned_addr learned my addr 172.21.15.7:0/2148772691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f236f7fe640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23700b6b20 0x7f23701f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f236ffff640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700b2470 0x7f23701574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.863+0000 7f236effd640 1 -- 172.21.15.7:0/2148772691 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23700b6b20 msgr2=0x7f23701f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f236effd640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23700b6b20 0x7f23701f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f236effd640 1 -- 172.21.15.7:0/2148772691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700b2470 msgr2=0x7f23701574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f236effd640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700b2470 0x7f23701574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f236effd640 1 -- 172.21.15.7:0/2148772691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23700b28f0 con 0x7f23700b6f00 2024-09-15T23:36:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f236ffff640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700b2470 0x7f23701574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:36:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f236effd640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23700b6f00 0x7f23700b1f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f23640072c0 tx=0x7f2364007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f2357fff640 1 -- 172.21.15.7:0/2148772691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364001660 con 0x7f23700b6f00 2024-09-15T23:36:57.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f2357fff640 1 -- 172.21.15.7:0/2148772691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2364018070 con 0x7f23700b6f00 2024-09-15T23:36:57.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23700c79f0 con 0x7f23700b6f00 2024-09-15T23:36:57.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f2357fff640 1 -- 172.21.15.7:0/2148772691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236400fb80 con 0x7f23700b6f00 2024-09-15T23:36:57.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.864+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23700c7fb0 con 0x7f23700b6f00 2024-09-15T23:36:57.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.865+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23700b38d0 con 0x7f23700b6f00 2024-09-15T23:36:57.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.865+0000 7f2357fff640 1 -- 172.21.15.7:0/2148772691 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f236400fd20 con 0x7f23700b6f00 2024-09-15T23:36:57.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.866+0000 7f2357fff640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f235c06c480 0x7f235c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:57.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.866+0000 7f2357fff640 1 -- 172.21.15.7:0/2148772691 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f236408b650 con 0x7f23700b6f00 2024-09-15T23:36:57.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.866+0000 7f236f7fe640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f235c06c480 0x7f235c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:57.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.866+0000 7f236f7fe640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f235c06c480 0x7f235c06e940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2360032000 tx=0x7f236002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:57.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:57.870+0000 7f2357fff640 1 -- 172.21.15.7:0/2148772691 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2364059b10 con 0x7f23700b6f00 2024-09-15T23:36:58.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.049+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f23700d2000 con 0x7f23700b6f00 2024-09-15T23:36:58.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.049+0000 7f2357fff640 1 -- 172.21.15.7:0/2148772691 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f2364059410 con 0x7f23700b6f00 2024-09-15T23:36:58.052 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-15T23:36:58.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f235c06c480 msgr2=0x7f235c06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f235c06c480 0x7f235c06e940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2360032000 tx=0x7f236002e000 comp rx=0 tx=0).stop 2024-09-15T23:36:58.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23700b6f00 msgr2=0x7f23700b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23700b6f00 0x7f23700b1f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f23640072c0 tx=0x7f2364007780 comp rx=0 tx=0).stop 2024-09-15T23:36:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 shutdown_connections 2024-09-15T23:36:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700b2470 0x7f23701574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23700b6f00 0x7f23700b1f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f235c06c480 0x7f235c06e940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 --2- 172.21.15.7:0/2148772691 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23700b6b20 0x7f23701f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 >> 172.21.15.7:0/2148772691 conn(0x7f23700b3f40 msgr2=0x7f23700b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 shutdown_connections 2024-09-15T23:36:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.052+0000 7f2375f18640 1 -- 172.21.15.7:0/2148772691 wait complete. 2024-09-15T23:36:58.065 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-15T23:36:58.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.169+0000 7f999d45e640 1 Processor -- start 2024-09-15T23:36:58.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.170+0000 7f999d45e640 1 -- start start 2024-09-15T23:36:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.170+0000 7f999d45e640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f999815e900 0x7f99980cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.170+0000 7f999d45e640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99980d21e0 con 0x7f99980d1020 2024-09-15T23:36:58.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.170+0000 7f999d45e640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99980d24a0 con 0x7f99980d3360 2024-09-15T23:36:58.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.170+0000 7f999d45e640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f99980d2bf0 con 0x7f999815e900 2024-09-15T23:36:58.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.170+0000 7f9996ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f99980d1020 0x7f99980d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56774/0 (socket says 172.21.15.7:56774) 2024-09-15T23:36:58.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.170+0000 7f9996ffd640 1 -- 172.21.15.7:0/2192475133 learned_addr learned my addr 172.21.15.7:0/2192475133 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:58.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f99967fc640 1 --2- 172.21.15.7:0/2192475133 >> v2:172.21.15.18:3300/0 conn(0x7f999815e900 0x7f99980cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1821735145 0 0) 0x7f99980d21e0 con 0x7f99980d1020 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9984003520 con 0x7f99980d1020 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1993020494 0 0) 0x7f9984003520 con 0x7f99980d1020 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 >> v2:172.21.15.18:3300/0 conn(0x7f999815e900 msgr2=0x7f99980cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 --2- 172.21.15.7:0/2192475133 >> v2:172.21.15.18:3300/0 conn(0x7f999815e900 0x7f99980cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 >> v1:172.21.15.18:6789/0 conn(0x7f99980d3360 legacy=0x7f99980cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99980d2780 con 0x7f99980d1020 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f99880019f0 con 0x7f99980d1020 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 >> v1:172.21.15.7:6789/0 conn(0x7f99980d1020 legacy=0x7f99980d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f9984003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9984004800 0x7f9984004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9984005210 0x7f998400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99980d2bf0 con 0x7f99840039b0 2024-09-15T23:36:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f998400c140 con 0x7f9984004800 2024-09-15T23:36:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f998400c340 con 0x7f9984005210 2024-09-15T23:36:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f9996ffd640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f9984003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f99967fc640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9984004800 0x7f9984004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.171+0000 7f99977fe640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9984005210 0x7f998400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f9996ffd640 1 -- 172.21.15.7:0/2192475133 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9984005210 msgr2=0x7f998400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f9996ffd640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9984005210 0x7f998400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f9996ffd640 1 -- 172.21.15.7:0/2192475133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9984004800 msgr2=0x7f9984004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f9996ffd640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9984004800 0x7f9984004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f9996ffd640 1 -- 172.21.15.7:0/2192475133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9984003520 con 0x7f99840039b0 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f99977fe640 1 -- 172.21.15.7:0/2192475133 reap_dead start 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f9996ffd640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f9984003df0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f998800a840 tx=0x7f9984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151eb55651ac05d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998800a460 con 0x7f99840039b0 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998802f070 con 0x7f99840039b0 2024-09-15T23:36:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.172+0000 7f9995ffb640 1 -- 172.21.15.7:0/2192475133 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998802bcf0 con 0x7f99840039b0 2024-09-15T23:36:58.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.173+0000 7f999d45e640 1 -- 172.21.15.7:0/2192475133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 msgr2=0x7f9984003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.173+0000 7f999d45e640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f9984003df0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f998800a840 tx=0x7f9984005830 comp rx=0 tx=0).stop 2024-09-15T23:36:58.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.173+0000 7f999d45e640 1 -- 172.21.15.7:0/2192475133 shutdown_connections 2024-09-15T23:36:58.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.173+0000 7f999d45e640 1 --2- 172.21.15.7:0/2192475133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f9984003df0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.173+0000 7f999d45e640 1 -- 172.21.15.7:0/2192475133 >> 172.21.15.7:0/2192475133 conn(0x7f999808bfd0 msgr2=0x7f99980d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:58.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.173+0000 7f999d45e640 1 -- 172.21.15.7:0/2192475133 shutdown_connections 2024-09-15T23:36:58.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.173+0000 7f999d45e640 1 -- 172.21.15.7:0/2192475133 wait complete. 2024-09-15T23:36:58.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.174+0000 7f999d45e640 1 Processor -- start 2024-09-15T23:36:58.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.174+0000 7f999d45e640 1 -- start start 2024-09-15T23:36:58.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.175+0000 7f999d45e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f999815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.175+0000 7f999d45e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f999815d9a0 0x7f99981e9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.175+0000 7f999d45e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99980b26f0 0x7f99980b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.175+0000 7f999d45e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99980b3850 con 0x7f99840039b0 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.175+0000 7f999d45e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99980b36d0 con 0x7f99980b26f0 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.175+0000 7f999d45e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f99980d1400 con 0x7f999815d9a0 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.175+0000 7f9996ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f999815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.175+0000 7f9996ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f999815ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60356/0 (socket says 172.21.15.7:60356) 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.175+0000 7f9996ffd640 1 -- 172.21.15.7:0/2068896018 learned_addr learned my addr 172.21.15.7:0/2068896018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f9996ffd640 1 -- 172.21.15.7:0/2068896018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f999815d9a0 msgr2=0x7f99981e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f99967fc640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f999815d9a0 0x7f99981e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f99977fe640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99980b26f0 0x7f99980b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f9996ffd640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f999815d9a0 0x7f99981e9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f9996ffd640 1 -- 172.21.15.7:0/2068896018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99980b26f0 msgr2=0x7f99980b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f9996ffd640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99980b26f0 0x7f99980b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f9996ffd640 1 -- 172.21.15.7:0/2068896018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99981e9db0 con 0x7f99840039b0 2024-09-15T23:36:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f9996ffd640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f999815ece0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f998800a840 tx=0x7f99980d2670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f998f7fe640 1 -- 172.21.15.7:0/2068896018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988044070 con 0x7f99840039b0 2024-09-15T23:36:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f998f7fe640 1 -- 172.21.15.7:0/2068896018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9988034670 con 0x7f99840039b0 2024-09-15T23:36:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f998f7fe640 1 -- 172.21.15.7:0/2068896018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998800a460 con 0x7f99840039b0 2024-09-15T23:36:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999815f370 con 0x7f99840039b0 2024-09-15T23:36:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.176+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999815f930 con 0x7f99840039b0 2024-09-15T23:36:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.177+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99980b30b0 con 0x7f99840039b0 2024-09-15T23:36:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.178+0000 7f998f7fe640 1 -- 172.21.15.7:0/2068896018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9988047070 con 0x7f99840039b0 2024-09-15T23:36:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.178+0000 7f998f7fe640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f997806c2e0 0x7f997806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.178+0000 7f998f7fe640 1 -- 172.21.15.7:0/2068896018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f99880ab670 con 0x7f99840039b0 2024-09-15T23:36:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.178+0000 7f99967fc640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f997806c2e0 0x7f997806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.179+0000 7f99967fc640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f997806c2e0 0x7f997806e7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99980b3420 tx=0x7f9980008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.182+0000 7f998f7fe640 1 -- 172.21.15.7:0/2068896018 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9988079450 con 0x7f99840039b0 2024-09-15T23:36:58.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.357+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9998068d80 con 0x7f99840039b0 2024-09-15T23:36:58.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.357+0000 7f998f7fe640 1 -- 172.21.15.7:0/2068896018 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f998807d750 con 0x7f99840039b0 2024-09-15T23:36:58.360 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-15T23:36:58.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f997806c2e0 msgr2=0x7f997806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f997806c2e0 0x7f997806e7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99980b3420 tx=0x7f9980008040 comp rx=0 tx=0).stop 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 msgr2=0x7f999815ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f999815ece0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f998800a840 tx=0x7f99980d2670 comp rx=0 tx=0).stop 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 shutdown_connections 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99980b26f0 0x7f99980b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f997806c2e0 0x7f997806e7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f999815d9a0 0x7f99981e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 --2- 172.21.15.7:0/2068896018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99840039b0 0x7f999815ece0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 >> 172.21.15.7:0/2068896018 conn(0x7f999808bfd0 msgr2=0x7f99980c9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 shutdown_connections 2024-09-15T23:36:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.359+0000 7f999d45e640 1 -- 172.21.15.7:0/2068896018 wait complete. 2024-09-15T23:36:58.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-15T23:36:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.476+0000 7f64c1d31640 1 Processor -- start 2024-09-15T23:36:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.476+0000 7f64c1d31640 1 -- start start 2024-09-15T23:36:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64c1d31640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc0af370 0x7f64bc06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64c1d31640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64bc0cb210 con 0x7f64bc0cadb0 2024-09-15T23:36:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64c1d31640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64bc0701a0 con 0x7f64bc06bc10 2024-09-15T23:36:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64c1d31640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f64bc06f570 con 0x7f64bc0af370 2024-09-15T23:36:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64baffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f64bc0cadb0 0x7f64bc0aec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56784/0 (socket says 172.21.15.7:56784) 2024-09-15T23:36:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64baffd640 1 -- 172.21.15.7:0/370267644 learned_addr learned my addr 172.21.15.7:0/370267644 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:36:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64bbfff640 1 --2- 172.21.15.7:0/370267644 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc0af370 0x7f64bc06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64ba7fc640 1 -- 172.21.15.7:0/370267644 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3132776398 0 0) 0x7f64bc0cb210 con 0x7f64bc0cadb0 2024-09-15T23:36:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64ba7fc640 1 -- 172.21.15.7:0/370267644 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64a4003520 con 0x7f64bc0cadb0 2024-09-15T23:36:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64ba7fc640 1 -- 172.21.15.7:0/370267644 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2735569708 0 0) 0x7f64a4003520 con 0x7f64bc0cadb0 2024-09-15T23:36:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64bbfff640 1 -- 172.21.15.7:0/370267644 >> v1:172.21.15.18:6789/0 conn(0x7f64bc06bc10 legacy=0x7f64bc06fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64bbfff640 1 -- 172.21.15.7:0/370267644 >> v1:172.21.15.7:6789/0 conn(0x7f64bc0cadb0 legacy=0x7f64bc0aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.477+0000 7f64bbfff640 1 -- 172.21.15.7:0/370267644 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc06f290 con 0x7f64bc0af370 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.478+0000 7f64bbfff640 1 --2- 172.21.15.7:0/370267644 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc0af370 0x7f64bc06bff0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009dc0 tx=0x7f64b402a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df550a4812db8808 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.478+0000 7f64ba7fc640 1 -- 172.21.15.7:0/370267644 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4035020 con 0x7f64bc0af370 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.478+0000 7f64ba7fc640 1 -- 172.21.15.7:0/370267644 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b4007540 con 0x7f64bc0af370 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.478+0000 7f64ba7fc640 1 -- 172.21.15.7:0/370267644 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0001670 con 0x7f64bc0af370 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.479+0000 7f64c1d31640 1 -- 172.21.15.7:0/370267644 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc0af370 msgr2=0x7f64bc06bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.479+0000 7f64c1d31640 1 --2- 172.21.15.7:0/370267644 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc0af370 0x7f64bc06bff0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009dc0 tx=0x7f64b402a580 comp rx=0 tx=0).stop 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.479+0000 7f64c1d31640 1 -- 172.21.15.7:0/370267644 shutdown_connections 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.479+0000 7f64c1d31640 1 --2- 172.21.15.7:0/370267644 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc0af370 0x7f64bc06bff0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.479+0000 7f64c1d31640 1 -- 172.21.15.7:0/370267644 >> 172.21.15.7:0/370267644 conn(0x7f64bc0b2fc0 msgr2=0x7f64bc0b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.479+0000 7f64c1d31640 1 -- 172.21.15.7:0/370267644 shutdown_connections 2024-09-15T23:36:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.479+0000 7f64c1d31640 1 -- 172.21.15.7:0/370267644 wait complete. 2024-09-15T23:36:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.480+0000 7f64c1d31640 1 Processor -- start 2024-09-15T23:36:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.480+0000 7f64c1d31640 1 -- start start 2024-09-15T23:36:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64c1d31640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc06bc10 0x7f64bc1f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64c1d31640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64bc0af370 0x7f64bc0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64c1d31640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64bc0cadb0 0x7f64bc155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64c1d31640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc0ca7e0 con 0x7f64bc0af370 2024-09-15T23:36:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64c1d31640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc0cb780 con 0x7f64bc0cadb0 2024-09-15T23:36:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64c1d31640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f64bc06c530 con 0x7f64bc06bc10 2024-09-15T23:36:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64baffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64bc0af370 0x7f64bc0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64baffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64bc0af370 0x7f64bc0b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60364/0 (socket says 172.21.15.7:60364) 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64baffd640 1 -- 172.21.15.7:0/2433434778 learned_addr learned my addr 172.21.15.7:0/2433434778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64bbfff640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64bc0cadb0 0x7f64bc155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.481+0000 7f64bb7fe640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc06bc10 0x7f64bc1f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64baffd640 1 -- 172.21.15.7:0/2433434778 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc06bc10 msgr2=0x7f64bc1f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64baffd640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc06bc10 0x7f64bc1f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64baffd640 1 -- 172.21.15.7:0/2433434778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64bc0cadb0 msgr2=0x7f64bc155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64baffd640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64bc0cadb0 0x7f64bc155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64baffd640 1 -- 172.21.15.7:0/2433434778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc0b1bf0 con 0x7f64bc0af370 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64baffd640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64bc0af370 0x7f64bc0b1450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f64b000c550 tx=0x7f64b000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64a3fff640 1 -- 172.21.15.7:0/2433434778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0018070 con 0x7f64bc0af370 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64a3fff640 1 -- 172.21.15.7:0/2433434778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b0007500 con 0x7f64bc0af370 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc0b1ed0 con 0x7f64bc0af370 2024-09-15T23:36:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64a3fff640 1 -- 172.21.15.7:0/2433434778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0004a20 con 0x7f64bc0af370 2024-09-15T23:36:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.482+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc156060 con 0x7f64bc0af370 2024-09-15T23:36:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.483+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc068480 con 0x7f64bc0af370 2024-09-15T23:36:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.483+0000 7f64a3fff640 1 -- 172.21.15.7:0/2433434778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f64b001b070 con 0x7f64bc0af370 2024-09-15T23:36:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.484+0000 7f64a3fff640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64a806c2b0 0x7f64a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.484+0000 7f64a3fff640 1 -- 172.21.15.7:0/2433434778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f64b008aa90 con 0x7f64bc0af370 2024-09-15T23:36:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.484+0000 7f64bb7fe640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64a806c2b0 0x7f64a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.485+0000 7f64bb7fe640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64a806c2b0 0x7f64a806e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f64ac001900 tx=0x7f64ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.489+0000 7f64a3fff640 1 -- 172.21.15.7:0/2433434778 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f64bc068480 con 0x7f64bc0af370 2024-09-15T23:36:58.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.663+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f64bc06c900 con 0x7f64bc0af370 2024-09-15T23:36:58.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.663+0000 7f64a3fff640 1 -- 172.21.15.7:0/2433434778 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f64b0058870 con 0x7f64bc0af370 2024-09-15T23:36:58.665 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-15T23:36:58.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64a806c2b0 msgr2=0x7f64a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64a806c2b0 0x7f64a806e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f64ac001900 tx=0x7f64ac009040 comp rx=0 tx=0).stop 2024-09-15T23:36:58.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64bc0af370 msgr2=0x7f64bc0b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64bc0af370 0x7f64bc0b1450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f64b000c550 tx=0x7f64b000ca10 comp rx=0 tx=0).stop 2024-09-15T23:36:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 shutdown_connections 2024-09-15T23:36:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64bc0cadb0 0x7f64bc155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64bc0af370 0x7f64bc0b1450 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64a806c2b0 0x7f64a806e770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 --2- 172.21.15.7:0/2433434778 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f64bc06bc10 0x7f64bc1f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 >> 172.21.15.7:0/2433434778 conn(0x7f64bc0b2fc0 msgr2=0x7f64bc0aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 shutdown_connections 2024-09-15T23:36:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.665+0000 7f64c1d31640 1 -- 172.21.15.7:0/2433434778 wait complete. 2024-09-15T23:36:58.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-09-15T23:36:58.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.779+0000 7fdc95c4c640 1 Processor -- start 2024-09-15T23:36:58.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.779+0000 7fdc95c4c640 1 -- start start 2024-09-15T23:36:58.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.780+0000 7fdc95c4c640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdc900ceff0 0x7fdc9006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.780+0000 7fdc95c4c640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9015ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.780+0000 7fdc95c4c640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc900ce270 con 0x7fdc9015aac0 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.780+0000 7fdc95c4c640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fdc9006fd90 con 0x7fdc900ceff0 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.780+0000 7fdc95c4c640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fdc9015ef60 con 0x7fdc9015e5f0 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8effd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdc900ceff0 0x7fdc9006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8ffff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9015ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8effd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdc900ceff0 0x7fdc9006f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:34156/0 (socket says 172.21.15.7:34156) 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8ffff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9015ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:53706/0 (socket says 172.21.15.7:53706) 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8effd640 1 -- 172.21.15.7:0/2964153720 learned_addr learned my addr 172.21.15.7:0/2964153720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8f7fe640 1 --1- 172.21.15.7:0/2964153720 >> v1:172.21.15.18:6789/0 conn(0x7fdc9015aac0 0x7fdc900b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:43078/0 (socket says 172.21.15.7:43078) 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8ffff640 1 -- 172.21.15.7:0/2964153720 >> v2:172.21.15.18:3300/0 conn(0x7fdc900ceff0 msgr2=0x7fdc9006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8ffff640 1 --2- 172.21.15.7:0/2964153720 >> v2:172.21.15.18:3300/0 conn(0x7fdc900ceff0 0x7fdc9006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8ffff640 1 -- 172.21.15.7:0/2964153720 >> v1:172.21.15.18:6789/0 conn(0x7fdc9015aac0 legacy=0x7fdc900b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8ffff640 1 -- 172.21.15.7:0/2964153720 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc900ce510 con 0x7fdc9015e5f0 2024-09-15T23:36:58.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.781+0000 7fdc8effd640 1 --2- 172.21.15.7:0/2964153720 >> v2:172.21.15.18:3300/0 conn(0x7fdc900ceff0 0x7fdc9006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.782+0000 7fdc8ffff640 1 --2- 172.21.15.7:0/2964153720 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9015ea20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009d90 tx=0x7fdc8802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1d5ff62e98e3f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.782+0000 7fdc8e7fc640 1 -- 172.21.15.7:0/2964153720 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88007740 con 0x7fdc9015e5f0 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.782+0000 7fdc8e7fc640 1 -- 172.21.15.7:0/2964153720 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc880078e0 con 0x7fdc9015e5f0 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.782+0000 7fdc8e7fc640 1 -- 172.21.15.7:0/2964153720 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88007cb0 con 0x7fdc9015e5f0 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.782+0000 7fdc95c4c640 1 -- 172.21.15.7:0/2964153720 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 msgr2=0x7fdc9015ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.782+0000 7fdc95c4c640 1 --2- 172.21.15.7:0/2964153720 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9015ea20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009d90 tx=0x7fdc8802a660 comp rx=0 tx=0).stop 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.783+0000 7fdc95c4c640 1 -- 172.21.15.7:0/2964153720 shutdown_connections 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.783+0000 7fdc95c4c640 1 --2- 172.21.15.7:0/2964153720 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9015ea20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.783+0000 7fdc95c4c640 1 --2- 172.21.15.7:0/2964153720 >> v2:172.21.15.18:3300/0 conn(0x7fdc900ceff0 0x7fdc9006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.783+0000 7fdc95c4c640 1 -- 172.21.15.7:0/2964153720 >> 172.21.15.7:0/2964153720 conn(0x7fdc900b21e0 msgr2=0x7fdc900b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.783+0000 7fdc95c4c640 1 -- 172.21.15.7:0/2964153720 shutdown_connections 2024-09-15T23:36:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.783+0000 7fdc95c4c640 1 -- 172.21.15.7:0/2964153720 wait complete. 2024-09-15T23:36:58.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.784+0000 7fdc95c4c640 1 Processor -- start 2024-09-15T23:36:58.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.784+0000 7fdc95c4c640 1 -- start start 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.784+0000 7fdc95c4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc900ceff0 0x7fdc90159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc95c4c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc9015aac0 0x7fdc9009af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc95c4c640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9009b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc95c4c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9015aea0 con 0x7fdc900ceff0 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc95c4c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdc900cead0 con 0x7fdc9015aac0 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc95c4c640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fdc900d05f0 con 0x7fdc9015e5f0 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc900ceff0 0x7fdc90159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc900ceff0 0x7fdc90159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60368/0 (socket says 172.21.15.7:60368) 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8f7fe640 1 -- 172.21.15.7:0/123978092 learned_addr learned my addr 172.21.15.7:0/123978092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8ffff640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9009b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8effd640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc9015aac0 0x7fdc9009af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8f7fe640 1 -- 172.21.15.7:0/123978092 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 msgr2=0x7fdc9009b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8f7fe640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9009b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8f7fe640 1 -- 172.21.15.7:0/123978092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc9015aac0 msgr2=0x7fdc9009af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8f7fe640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc9015aac0 0x7fdc9009af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8f7fe640 1 -- 172.21.15.7:0/123978092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc90158050 con 0x7fdc900ceff0 2024-09-15T23:36:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc8f7fe640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc900ceff0 0x7fdc90159890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c450 tx=0x7fdc8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc77fff640 1 -- 172.21.15.7:0/123978092 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80018070 con 0x7fdc900ceff0 2024-09-15T23:36:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc77fff640 1 -- 172.21.15.7:0/123978092 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc800074a0 con 0x7fdc900ceff0 2024-09-15T23:36:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9009a1b0 con 0x7fdc900ceff0 2024-09-15T23:36:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc77fff640 1 -- 172.21.15.7:0/123978092 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80004a20 con 0x7fdc900ceff0 2024-09-15T23:36:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.785+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9009a770 con 0x7fdc900ceff0 2024-09-15T23:36:58.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.786+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc9006caa0 con 0x7fdc900ceff0 2024-09-15T23:36:58.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.787+0000 7fdc77fff640 1 -- 172.21.15.7:0/123978092 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdc80004bc0 con 0x7fdc900ceff0 2024-09-15T23:36:58.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.787+0000 7fdc77fff640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fdc7c06c2b0 0x7fdc7c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:36:58.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.788+0000 7fdc77fff640 1 -- 172.21.15.7:0/123978092 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdc8008a8e0 con 0x7fdc900ceff0 2024-09-15T23:36:58.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.788+0000 7fdc8effd640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fdc7c06c2b0 0x7fdc7c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:36:58.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.788+0000 7fdc8effd640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fdc7c06c2b0 0x7fdc7c06e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400a7d0 tx=0x7fdc8400a350 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:36:58.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:58.792+0000 7fdc77fff640 1 -- 172.21.15.7:0/123978092 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc9006caa0 con 0x7fdc900ceff0 2024-09-15T23:36:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.036+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) -- 0x7fdc900aee20 con 0x7fdc900ceff0 2024-09-15T23:36:59.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.926+0000 7fdc77fff640 1 -- 172.21.15.7:0/123978092 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v56) ==== 158+0+0 (secure 0 0 0) 0x7fdc80007650 con 0x7fdc900ceff0 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.928+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fdc7c06c2b0 msgr2=0x7fdc7c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fdc7c06c2b0 0x7fdc7c06e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400a7d0 tx=0x7fdc8400a350 comp rx=0 tx=0).stop 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc900ceff0 msgr2=0x7fdc90159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc900ceff0 0x7fdc90159890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c450 tx=0x7fdc8000c910 comp rx=0 tx=0).stop 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 shutdown_connections 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fdc7c06c2b0 0x7fdc7c06e770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc9015e5f0 0x7fdc9009b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc9015aac0 0x7fdc9009af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 --2- 172.21.15.7:0/123978092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc900ceff0 0x7fdc90159890 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 >> 172.21.15.7:0/123978092 conn(0x7fdc900b21e0 msgr2=0x7fdc900cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 shutdown_connections 2024-09-15T23:36:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:36:59.929+0000 7fdc95c4c640 1 -- 172.21.15.7:0/123978092 wait complete. 2024-09-15T23:36:59.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-15T23:37:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.054+0000 7f3142030640 1 Processor -- start 2024-09-15T23:37:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.054+0000 7f3142030640 1 -- start start 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.055+0000 7f3142030640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f313c162320 0x7f313c162780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.055+0000 7f3142030640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313c162090 con 0x7f313c161cb0 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.055+0000 7f3142030640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313c0b3f30 con 0x7f313c0b4fd0 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.055+0000 7f3142030640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f313c162cc0 con 0x7f313c162320 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.055+0000 7f313b7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f313c161cb0 0x7f313c0b1330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56790/0 (socket says 172.21.15.7:56790) 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.055+0000 7f313b7fe640 1 -- 172.21.15.7:0/2069855029 learned_addr learned my addr 172.21.15.7:0/2069855029 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.055+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 382742937 0 0) 0x7f313c162090 con 0x7f313c161cb0 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.055+0000 7f313affd640 1 --2- 172.21.15.7:0/2069855029 >> v2:172.21.15.18:3300/0 conn(0x7f313c162320 0x7f313c162780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.055+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3124003520 con 0x7f313c161cb0 2024-09-15T23:37:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4051851211 0 0) 0x7f3124003520 con 0x7f313c161cb0 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 >> v2:172.21.15.18:3300/0 conn(0x7f313c162320 msgr2=0x7f313c162780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 --2- 172.21.15.7:0/2069855029 >> v2:172.21.15.18:3300/0 conn(0x7f313c162320 0x7f313c162780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 >> v1:172.21.15.18:6789/0 conn(0x7f313c0b4fd0 legacy=0x7f313c0b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c0b4210 con 0x7f313c161cb0 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f312c001670 con 0x7f313c161cb0 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 >> v1:172.21.15.7:6789/0 conn(0x7f313c161cb0 legacy=0x7f313c0b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31240039b0 0x7f3124003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124004800 0x7f3124004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f312400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f312400c030 con 0x7f31240051c0 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f312400c1e0 con 0x7f3124004800 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f312400c3e0 con 0x7f31240039b0 2024-09-15T23:37:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313bfff640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f312400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313affd640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124004800 0x7f3124004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313b7fe640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31240039b0 0x7f3124003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313bfff640 1 -- 172.21.15.7:0/2069855029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31240039b0 msgr2=0x7f3124003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313bfff640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31240039b0 0x7f3124003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313bfff640 1 -- 172.21.15.7:0/2069855029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124004800 msgr2=0x7f3124004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313bfff640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124004800 0x7f3124004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.056+0000 7f313bfff640 1 -- 172.21.15.7:0/2069855029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3124003520 con 0x7f31240051c0 2024-09-15T23:37:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.057+0000 7f313bfff640 1 -- 172.21.15.7:0/2069855029 reap_dead start 2024-09-15T23:37:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.057+0000 7f313bfff640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f312400b9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3134001e50 tx=0x7f3124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef24d666fa607636 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.057+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134001a70 con 0x7f31240051c0 2024-09-15T23:37:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.057+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3134003970 con 0x7f31240051c0 2024-09-15T23:37:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.057+0000 7f313a7fc640 1 -- 172.21.15.7:0/2069855029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134003d60 con 0x7f31240051c0 2024-09-15T23:37:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.058+0000 7f3142030640 1 -- 172.21.15.7:0/2069855029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 msgr2=0x7f312400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.058+0000 7f3142030640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f312400b9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3134001e50 tx=0x7f3124005830 comp rx=0 tx=0).stop 2024-09-15T23:37:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.058+0000 7f3142030640 1 -- 172.21.15.7:0/2069855029 shutdown_connections 2024-09-15T23:37:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.058+0000 7f3142030640 1 --2- 172.21.15.7:0/2069855029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f312400b9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.058+0000 7f3142030640 1 -- 172.21.15.7:0/2069855029 >> 172.21.15.7:0/2069855029 conn(0x7f313c08bfd0 msgr2=0x7f313c0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.058+0000 7f3142030640 1 -- 172.21.15.7:0/2069855029 shutdown_connections 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.058+0000 7f3142030640 1 -- 172.21.15.7:0/2069855029 wait complete. 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.059+0000 7f3142030640 1 Processor -- start 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.059+0000 7f3142030640 1 -- start start 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f3142030640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f313c0b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f3142030640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c0c7510 0x7f313c149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f3142030640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f313c1f4b40 0x7f313c1f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f3142030640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f313c0b47d0 con 0x7f31240051c0 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f3142030640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f313c0b4b40 con 0x7f313c0c7510 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f3142030640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f313c0b3bc0 con 0x7f313c1f4b40 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f313c0b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f313c0b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60380/0 (socket says 172.21.15.7:60380) 2024-09-15T23:37:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313b7fe640 1 -- 172.21.15.7:0/3234282890 learned_addr learned my addr 172.21.15.7:0/3234282890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313affd640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c0c7510 0x7f313c149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313bfff640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f313c1f4b40 0x7f313c1f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313b7fe640 1 -- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f313c1f4b40 msgr2=0x7f313c1f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313b7fe640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f313c1f4b40 0x7f313c1f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313b7fe640 1 -- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c0c7510 msgr2=0x7f313c149100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313b7fe640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c0c7510 0x7f313c149100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313b7fe640 1 -- 172.21.15.7:0/3234282890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c149640 con 0x7f31240051c0 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313bfff640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f313c1f4b40 0x7f313c1f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.060+0000 7f313affd640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c0c7510 0x7f313c149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.061+0000 7f313b7fe640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f313c0b77d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f312c007290 tx=0x7f312c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.061+0000 7f3123fff640 1 -- 172.21.15.7:0/3234282890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c00f960 con 0x7f31240051c0 2024-09-15T23:37:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.061+0000 7f3123fff640 1 -- 172.21.15.7:0/3234282890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312c018070 con 0x7f31240051c0 2024-09-15T23:37:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.061+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c0c7e70 con 0x7f31240051c0 2024-09-15T23:37:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.061+0000 7f3123fff640 1 -- 172.21.15.7:0/3234282890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c00fd00 con 0x7f31240051c0 2024-09-15T23:37:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.061+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c0b7db0 con 0x7f31240051c0 2024-09-15T23:37:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.062+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c0c99d0 con 0x7f31240051c0 2024-09-15T23:37:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.062+0000 7f3123fff640 1 -- 172.21.15.7:0/3234282890 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f312c01d460 con 0x7f31240051c0 2024-09-15T23:37:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.063+0000 7f3123fff640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f312806c2b0 0x7f312806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.063+0000 7f3123fff640 1 -- 172.21.15.7:0/3234282890 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f312c08b800 con 0x7f31240051c0 2024-09-15T23:37:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.063+0000 7f313affd640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f312806c2b0 0x7f312806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.063+0000 7f313affd640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f312806c2b0 0x7f312806e770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f313c146530 tx=0x7f3130008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:00.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.067+0000 7f3123fff640 1 -- 172.21.15.7:0/3234282890 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f313c149640 con 0x7f31240051c0 2024-09-15T23:37:00.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.244+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f313c068fa0 con 0x7f31240051c0 2024-09-15T23:37:00.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.247+0000 7f3123fff640 1 -- 172.21.15.7:0/3234282890 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f312c059c30 con 0x7f31240051c0 2024-09-15T23:37:00.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f312806c2b0 msgr2=0x7f312806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f312806c2b0 0x7f312806e770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f313c146530 tx=0x7f3130008040 comp rx=0 tx=0).stop 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 msgr2=0x7f313c0b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f313c0b77d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f312c007290 tx=0x7f312c007750 comp rx=0 tx=0).stop 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 shutdown_connections 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f312806c2b0 0x7f312806e770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f313c1f4b40 0x7f313c1f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c0c7510 0x7f313c149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 --2- 172.21.15.7:0/3234282890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31240051c0 0x7f313c0b77d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 >> 172.21.15.7:0/3234282890 conn(0x7f313c08bfd0 msgr2=0x7f313c15fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 shutdown_connections 2024-09-15T23:37:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.249+0000 7f3142030640 1 -- 172.21.15.7:0/3234282890 wait complete. 2024-09-15T23:37:00.263 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-15T23:37:00.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-15T23:37:00.309 DEBUG:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-15T23:37:00.351 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-15T23:37:00.351 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:00.351 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:00.367 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:00.368 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:00.424 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:00.424 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:00.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-15T23:37:00.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.626+0000 7ffa232af640 1 Processor -- start 2024-09-15T23:37:00.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.626+0000 7ffa232af640 1 -- start start 2024-09-15T23:37:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa232af640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c15e920 0x7ffa1c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa232af640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa1c0b36f0 0x7ffa1c0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa232af640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ffa1c09aad0 0x7ffa1c06f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa232af640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c0aea30 con 0x7ffa1c0b36f0 2024-09-15T23:37:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa232af640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c070da0 con 0x7ffa1c09aad0 2024-09-15T23:37:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa232af640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c0aec60 con 0x7ffa1c15e920 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2084a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa1c0b36f0 0x7ffa1c0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2084a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa1c0b36f0 0x7ffa1c0b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60382/0 (socket says 172.21.15.7:60382) 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2084a640 1 -- 172.21.15.7:0/2640569435 learned_addr learned my addr 172.21.15.7:0/2640569435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2104b640 1 --2- 172.21.15.7:0/2640569435 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c15e920 0x7ffa1c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2184c640 1 --2- 172.21.15.7:0/2640569435 >> v2:172.21.15.18:3300/0 conn(0x7ffa1c09aad0 0x7ffa1c06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2084a640 1 -- 172.21.15.7:0/2640569435 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c15e920 msgr2=0x7ffa1c0b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2084a640 1 --2- 172.21.15.7:0/2640569435 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c15e920 0x7ffa1c0b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2084a640 1 -- 172.21.15.7:0/2640569435 >> v2:172.21.15.18:3300/0 conn(0x7ffa1c09aad0 msgr2=0x7ffa1c06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2084a640 1 --2- 172.21.15.7:0/2640569435 >> v2:172.21.15.18:3300/0 conn(0x7ffa1c09aad0 0x7ffa1c06f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2084a640 1 -- 172.21.15.7:0/2640569435 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0aef40 con 0x7ffa1c0b36f0 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2104b640 1 --2- 172.21.15.7:0/2640569435 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c15e920 0x7ffa1c0b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.628+0000 7ffa2184c640 1 --2- 172.21.15.7:0/2640569435 >> v2:172.21.15.18:3300/0 conn(0x7ffa1c09aad0 0x7ffa1c06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.629+0000 7ffa2084a640 1 --2- 172.21.15.7:0/2640569435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c0b36f0 0x7ffa1c0b53b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c06e7c0 tx=0x7ffa1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747ea6b3fbec0bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.629+0000 7ffa0ffff640 1 -- 172.21.15.7:0/2640569435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10037070 con 0x7ffa1c0b36f0 2024-09-15T23:37:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.629+0000 7ffa0ffff640 1 -- 172.21.15.7:0/2640569435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa10032440 con 0x7ffa1c0b36f0 2024-09-15T23:37:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.629+0000 7ffa0ffff640 1 -- 172.21.15.7:0/2640569435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10032810 con 0x7ffa1c0b36f0 2024-09-15T23:37:00.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.630+0000 7ffa232af640 1 -- 172.21.15.7:0/2640569435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c0b36f0 msgr2=0x7ffa1c0b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.630+0000 7ffa232af640 1 --2- 172.21.15.7:0/2640569435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c0b36f0 0x7ffa1c0b53b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c06e7c0 tx=0x7ffa1002a270 comp rx=0 tx=0).stop 2024-09-15T23:37:00.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.630+0000 7ffa232af640 1 -- 172.21.15.7:0/2640569435 shutdown_connections 2024-09-15T23:37:00.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.630+0000 7ffa232af640 1 --2- 172.21.15.7:0/2640569435 >> v2:172.21.15.18:3300/0 conn(0x7ffa1c09aad0 0x7ffa1c06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.630+0000 7ffa232af640 1 --2- 172.21.15.7:0/2640569435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c0b36f0 0x7ffa1c0b53b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.630+0000 7ffa232af640 1 --2- 172.21.15.7:0/2640569435 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c15e920 0x7ffa1c0b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.630+0000 7ffa232af640 1 -- 172.21.15.7:0/2640569435 >> 172.21.15.7:0/2640569435 conn(0x7ffa1c0734b0 msgr2=0x7ffa1c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:00.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.630+0000 7ffa232af640 1 -- 172.21.15.7:0/2640569435 shutdown_connections 2024-09-15T23:37:00.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.630+0000 7ffa232af640 1 -- 172.21.15.7:0/2640569435 wait complete. 2024-09-15T23:37:00.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.631+0000 7ffa232af640 1 Processor -- start 2024-09-15T23:37:00.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.631+0000 7ffa232af640 1 -- start start 2024-09-15T23:37:00.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa232af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c09aad0 0x7ffa1c09b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa232af640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c0b36f0 0x7ffa1c140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa232af640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c15e920 0x7ffa1c151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa232af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c15ed00 con 0x7ffa1c09aad0 2024-09-15T23:37:00.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa232af640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c070a30 con 0x7ffa1c15e920 2024-09-15T23:37:00.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa232af640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c15ee80 con 0x7ffa1c0b36f0 2024-09-15T23:37:00.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2104b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c09aad0 0x7ffa1c09b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2104b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c09aad0 0x7ffa1c09b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60392/0 (socket says 172.21.15.7:60392) 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2104b640 1 -- 172.21.15.7:0/955020089 learned_addr learned my addr 172.21.15.7:0/955020089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2184c640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c15e920 0x7ffa1c151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2084a640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c0b36f0 0x7ffa1c140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2104b640 1 -- 172.21.15.7:0/955020089 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c0b36f0 msgr2=0x7ffa1c140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2104b640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c0b36f0 0x7ffa1c140bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2104b640 1 -- 172.21.15.7:0/955020089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c15e920 msgr2=0x7ffa1c151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2104b640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c15e920 0x7ffa1c151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.632+0000 7ffa2104b640 1 -- 172.21.15.7:0/955020089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c14c540 con 0x7ffa1c09aad0 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.633+0000 7ffa2104b640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c09aad0 0x7ffa1c09b3d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800e810 tx=0x7ffa1800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.633+0000 7ffa0dffb640 1 -- 172.21.15.7:0/955020089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1800f040 con 0x7ffa1c09aad0 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.633+0000 7ffa0dffb640 1 -- 172.21.15.7:0/955020089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa18004030 con 0x7ffa1c09aad0 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.633+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c14c820 con 0x7ffa1c09aad0 2024-09-15T23:37:00.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.633+0000 7ffa0dffb640 1 -- 172.21.15.7:0/955020089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18004400 con 0x7ffa1c09aad0 2024-09-15T23:37:00.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.633+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c141100 con 0x7ffa1c09aad0 2024-09-15T23:37:00.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.634+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1c0b4e70 con 0x7ffa1c09aad0 2024-09-15T23:37:00.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.634+0000 7ffa0dffb640 1 -- 172.21.15.7:0/955020089 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffa1800c6d0 con 0x7ffa1c09aad0 2024-09-15T23:37:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.635+0000 7ffa0dffb640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ffa0006c2b0 0x7ffa0006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.635+0000 7ffa0dffb640 1 -- 172.21.15.7:0/955020089 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffa1808b450 con 0x7ffa1c09aad0 2024-09-15T23:37:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.635+0000 7ffa2084a640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ffa0006c2b0 0x7ffa0006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.635+0000 7ffa2084a640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ffa0006c2b0 0x7ffa0006e770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa10034000 tx=0x7ffa1002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:00.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.639+0000 7ffa0dffb640 1 -- 172.21.15.7:0/955020089 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffa18017020 con 0x7ffa1c09aad0 2024-09-15T23:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.828+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffa1c06fe30 con 0x7ffa1c09aad0 2024-09-15T23:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.828+0000 7ffa0dffb640 1 -- 172.21.15.7:0/955020089 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7ffa180592c0 con 0x7ffa1c09aad0 2024-09-15T23:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-15T23:37:00.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ffa0006c2b0 msgr2=0x7ffa0006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ffa0006c2b0 0x7ffa0006e770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa10034000 tx=0x7ffa1002e000 comp rx=0 tx=0).stop 2024-09-15T23:37:00.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c09aad0 msgr2=0x7ffa1c09b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:00.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c09aad0 0x7ffa1c09b3d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800e810 tx=0x7ffa1800ecd0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 shutdown_connections 2024-09-15T23:37:00.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c15e920 0x7ffa1c151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ffa0006c2b0 0x7ffa0006e770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ffa1c0b36f0 0x7ffa1c140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 --2- 172.21.15.7:0/955020089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa1c09aad0 0x7ffa1c09b3d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:00.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 >> 172.21.15.7:0/955020089 conn(0x7ffa1c0734b0 msgr2=0x7ffa1c15d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:00.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 shutdown_connections 2024-09-15T23:37:00.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:00.830+0000 7ffa232af640 1 -- 172.21.15.7:0/955020089 wait complete. 2024-09-15T23:37:00.843 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-15T23:37:00.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:00.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-15T23:37:00.871 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:37:00.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:00.872 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-15T23:37:00.928 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:00.928 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:00.949 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:00.949 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:01.008 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:01.008 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:01.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-15T23:37:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.170+0000 7fbf76253640 1 Processor -- start 2024-09-15T23:37:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.170+0000 7fbf76253640 1 -- start start 2024-09-15T23:37:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf76253640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70159e30 0x7fbf7006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf76253640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbf7006f520 0x7fbf7015d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf76253640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7006e480 con 0x7fbf700ad140 2024-09-15T23:37:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf76253640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fbf7006f940 con 0x7fbf7006f520 2024-09-15T23:37:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf76253640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7006e760 con 0x7fbf70159e30 2024-09-15T23:37:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70159e30 0x7fbf7006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf6effd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbf7006f520 0x7fbf7015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf6ffff640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fbf700ad140 0x7fbf700b3160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:43088/0 (socket says 172.21.15.7:43088) 2024-09-15T23:37:01.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70159e30 0x7fbf7006dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:53758/0 (socket says 172.21.15.7:53758) 2024-09-15T23:37:01.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.171+0000 7fbf6ffff640 1 -- 172.21.15.7:0/11349837 learned_addr learned my addr 172.21.15.7:0/11349837 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:01.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.172+0000 7fbf6f7fe640 1 -- 172.21.15.7:0/11349837 >> v2:172.21.15.18:3300/0 conn(0x7fbf7006f520 msgr2=0x7fbf7015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:01.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.172+0000 7fbf6f7fe640 1 --2- 172.21.15.7:0/11349837 >> v2:172.21.15.18:3300/0 conn(0x7fbf7006f520 0x7fbf7015d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.172+0000 7fbf6f7fe640 1 -- 172.21.15.7:0/11349837 >> v1:172.21.15.18:6789/0 conn(0x7fbf700ad140 legacy=0x7fbf700b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.172+0000 7fbf6f7fe640 1 -- 172.21.15.7:0/11349837 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7006ea40 con 0x7fbf70159e30 2024-09-15T23:37:01.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.172+0000 7fbf6f7fe640 1 --2- 172.21.15.7:0/11349837 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70159e30 0x7fbf7006dea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbf600097a0 tx=0x7fbf6002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c3216c5c854475 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:01.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf6e7fc640 1 -- 172.21.15.7:0/11349837 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60037070 con 0x7fbf70159e30 2024-09-15T23:37:01.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf6e7fc640 1 -- 172.21.15.7:0/11349837 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf60032440 con 0x7fbf70159e30 2024-09-15T23:37:01.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf6e7fc640 1 -- 172.21.15.7:0/11349837 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60032810 con 0x7fbf70159e30 2024-09-15T23:37:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf76253640 1 -- 172.21.15.7:0/11349837 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70159e30 msgr2=0x7fbf7006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf76253640 1 --2- 172.21.15.7:0/11349837 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70159e30 0x7fbf7006dea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbf600097a0 tx=0x7fbf6002a180 comp rx=0 tx=0).stop 2024-09-15T23:37:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf76253640 1 -- 172.21.15.7:0/11349837 shutdown_connections 2024-09-15T23:37:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf76253640 1 --2- 172.21.15.7:0/11349837 >> v2:172.21.15.18:3300/0 conn(0x7fbf7006f520 0x7fbf7015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf76253640 1 --2- 172.21.15.7:0/11349837 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70159e30 0x7fbf7006dea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf76253640 1 -- 172.21.15.7:0/11349837 >> 172.21.15.7:0/11349837 conn(0x7fbf700b1e60 msgr2=0x7fbf700b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf76253640 1 -- 172.21.15.7:0/11349837 shutdown_connections 2024-09-15T23:37:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.173+0000 7fbf76253640 1 -- 172.21.15.7:0/11349837 wait complete. 2024-09-15T23:37:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.175+0000 7fbf76253640 1 Processor -- start 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.175+0000 7fbf76253640 1 -- start start 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.175+0000 7fbf76253640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7006f520 0x7fbf70099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.175+0000 7fbf76253640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf700ad140 0x7fbf7015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.175+0000 7fbf76253640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf70159e30 0x7fbf7015b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.175+0000 7fbf76253640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7015a210 con 0x7fbf7006f520 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.175+0000 7fbf76253640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7006f000 con 0x7fbf70159e30 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.175+0000 7fbf76253640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7015a390 con 0x7fbf700ad140 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7006f520 0x7fbf70099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7006f520 0x7fbf70099860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60394/0 (socket says 172.21.15.7:60394) 2024-09-15T23:37:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6f7fe640 1 -- 172.21.15.7:0/1047200621 learned_addr learned my addr 172.21.15.7:0/1047200621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6effd640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf700ad140 0x7fbf7015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6ffff640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf70159e30 0x7fbf7015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6f7fe640 1 -- 172.21.15.7:0/1047200621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf700ad140 msgr2=0x7fbf7015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6f7fe640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf700ad140 0x7fbf7015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6f7fe640 1 -- 172.21.15.7:0/1047200621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf70159e30 msgr2=0x7fbf7015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6f7fe640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf70159e30 0x7fbf7015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6f7fe640 1 -- 172.21.15.7:0/1047200621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7014ee60 con 0x7fbf7006f520 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf6f7fe640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7006f520 0x7fbf70099860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf60000c00 tx=0x7fbf600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf67fff640 1 -- 172.21.15.7:0/1047200621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60037040 con 0x7fbf7006f520 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf67fff640 1 -- 172.21.15.7:0/1047200621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf60042070 con 0x7fbf7006f520 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7014f140 con 0x7fbf7006f520 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf67fff640 1 -- 172.21.15.7:0/1047200621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60030920 con 0x7fbf7006f520 2024-09-15T23:37:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.176+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7014f9e0 con 0x7fbf7006f520 2024-09-15T23:37:01.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.178+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7014f310 con 0x7fbf7006f520 2024-09-15T23:37:01.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.178+0000 7fbf67fff640 1 -- 172.21.15.7:0/1047200621 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbf60030ac0 con 0x7fbf7006f520 2024-09-15T23:37:01.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.178+0000 7fbf67fff640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbf5006c2b0 0x7fbf5006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.178+0000 7fbf67fff640 1 -- 172.21.15.7:0/1047200621 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbf600ac0b0 con 0x7fbf7006f520 2024-09-15T23:37:01.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.178+0000 7fbf6effd640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbf5006c2b0 0x7fbf5006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.179+0000 7fbf6effd640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbf5006c2b0 0x7fbf5006e770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf70156c10 tx=0x7fbf58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.187+0000 7fbf67fff640 1 -- 172.21.15.7:0/1047200621 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbf6007a4e0 con 0x7fbf7006f520 2024-09-15T23:37:01.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.396+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbf700b2240 con 0x7fbf7006f520 2024-09-15T23:37:01.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.397+0000 7fbf67fff640 1 -- 172.21.15.7:0/1047200621 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fbf700b2240 con 0x7fbf7006f520 2024-09-15T23:37:01.399 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-15T23:37:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbf5006c2b0 msgr2=0x7fbf5006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbf5006c2b0 0x7fbf5006e770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf70156c10 tx=0x7fbf58008040 comp rx=0 tx=0).stop 2024-09-15T23:37:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7006f520 msgr2=0x7fbf70099860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7006f520 0x7fbf70099860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf60000c00 tx=0x7fbf600046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 shutdown_connections 2024-09-15T23:37:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf70159e30 0x7fbf7015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbf5006c2b0 0x7fbf5006e770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf700ad140 0x7fbf7015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 --2- 172.21.15.7:0/1047200621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7006f520 0x7fbf70099860 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 >> 172.21.15.7:0/1047200621 conn(0x7fbf700b1e60 msgr2=0x7fbf700afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 shutdown_connections 2024-09-15T23:37:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.399+0000 7fbf76253640 1 -- 172.21.15.7:0/1047200621 wait complete. 2024-09-15T23:37:01.411 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-15T23:37:01.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:01.412 DEBUG:teuthology.orchestra.run.smithi018:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-15T23:37:01.444 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:37:01.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:01.444 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-15T23:37:01.516 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:01.532 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:01.549 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:01.549 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:01.605 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:01.605 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:01.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-15T23:37:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.804+0000 7f7d23865640 1 Processor -- start 2024-09-15T23:37:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.804+0000 7f7d23865640 1 -- start start 2024-09-15T23:37:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.805+0000 7f7d23865640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7d1c159e30 0x7f7d1c06dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.805+0000 7f7d23865640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c06f520 0x7f7d1c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.805+0000 7f7d23865640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d1c06e480 con 0x7f7d1c0ad140 2024-09-15T23:37:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.805+0000 7f7d23865640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7d1c06f940 con 0x7f7d1c159e30 2024-09-15T23:37:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.805+0000 7f7d23865640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1c06e760 con 0x7f7d1c06f520 2024-09-15T23:37:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.805+0000 7f7d20e00640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c06f520 0x7f7d1c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.805+0000 7f7d20e00640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c06f520 0x7f7d1c15d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:53788/0 (socket says 172.21.15.7:53788) 2024-09-15T23:37:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.805+0000 7f7d20e00640 1 -- 172.21.15.7:0/2988049588 learned_addr learned my addr 172.21.15.7:0/2988049588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.805+0000 7f7d21601640 1 --2- 172.21.15.7:0/2988049588 >> v2:172.21.15.18:3300/0 conn(0x7f7d1c159e30 0x7f7d1c06dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.806+0000 7f7d21601640 1 -- 172.21.15.7:0/2988049588 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c06f520 msgr2=0x7f7d1c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.806+0000 7f7d21601640 1 --2- 172.21.15.7:0/2988049588 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c06f520 0x7f7d1c15d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.806+0000 7f7d21601640 1 -- 172.21.15.7:0/2988049588 >> v1:172.21.15.18:6789/0 conn(0x7f7d1c0ad140 legacy=0x7f7d1c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.806+0000 7f7d21601640 1 -- 172.21.15.7:0/2988049588 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c06ea40 con 0x7f7d1c159e30 2024-09-15T23:37:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.806+0000 7f7d20e00640 1 --2- 172.21.15.7:0/2988049588 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c06f520 0x7f7d1c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.806+0000 7f7d21601640 1 --2- 172.21.15.7:0/2988049588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d1c159e30 0x7f7d1c06dea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d180097a0 tx=0x7f7d1802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ab36f4cc9d352c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d13fff640 1 -- 172.21.15.7:0/2988049588 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18037070 con 0x7f7d1c159e30 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d13fff640 1 -- 172.21.15.7:0/2988049588 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d18032440 con 0x7f7d1c159e30 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d13fff640 1 -- 172.21.15.7:0/2988049588 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18032810 con 0x7f7d1c159e30 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d23865640 1 -- 172.21.15.7:0/2988049588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d1c159e30 msgr2=0x7f7d1c06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d23865640 1 --2- 172.21.15.7:0/2988049588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d1c159e30 0x7f7d1c06dea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d180097a0 tx=0x7f7d1802a180 comp rx=0 tx=0).stop 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d23865640 1 -- 172.21.15.7:0/2988049588 shutdown_connections 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d23865640 1 --2- 172.21.15.7:0/2988049588 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c06f520 0x7f7d1c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d23865640 1 --2- 172.21.15.7:0/2988049588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d1c159e30 0x7f7d1c06dea0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d23865640 1 -- 172.21.15.7:0/2988049588 >> 172.21.15.7:0/2988049588 conn(0x7f7d1c0b1e60 msgr2=0x7f7d1c0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d23865640 1 -- 172.21.15.7:0/2988049588 shutdown_connections 2024-09-15T23:37:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.807+0000 7f7d23865640 1 -- 172.21.15.7:0/2988049588 wait complete. 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.809+0000 7f7d23865640 1 Processor -- start 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.809+0000 7f7d23865640 1 -- start start 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.809+0000 7f7d23865640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d1c06f520 0x7f7d1c1557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.809+0000 7f7d23865640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c0ad140 0x7f7d1c0b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.809+0000 7f7d23865640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c159e30 0x7f7d1c14f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.809+0000 7f7d23865640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c15a210 con 0x7f7d1c0ad140 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.809+0000 7f7d23865640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c06f000 con 0x7f7d1c06f520 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.809+0000 7f7d23865640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1c15a390 con 0x7f7d1c159e30 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.809+0000 7f7d20e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c0ad140 0x7f7d1c0b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d20e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c0ad140 0x7f7d1c0b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60402/0 (socket says 172.21.15.7:60402) 2024-09-15T23:37:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d20e00640 1 -- 172.21.15.7:0/781292674 learned_addr learned my addr 172.21.15.7:0/781292674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d21601640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d1c06f520 0x7f7d1c1557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d21e02640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c159e30 0x7f7d1c14f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d20e00640 1 -- 172.21.15.7:0/781292674 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c159e30 msgr2=0x7f7d1c14f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d20e00640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c159e30 0x7f7d1c14f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d20e00640 1 -- 172.21.15.7:0/781292674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d1c06f520 msgr2=0x7f7d1c1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d20e00640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d1c06f520 0x7f7d1c1557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d20e00640 1 -- 172.21.15.7:0/781292674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c1f5ab0 con 0x7f7d1c0ad140 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d21e02640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c159e30 0x7f7d1c14f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d20e00640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c0ad140 0x7f7d1c0b3f70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00ea00 tx=0x7f7d0c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d11ffb640 1 -- 172.21.15.7:0/781292674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c017070 con 0x7f7d1c0ad140 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d11ffb640 1 -- 172.21.15.7:0/781292674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c00ca00 con 0x7f7d1c0ad140 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c1f5d70 con 0x7f7d1c0ad140 2024-09-15T23:37:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d11ffb640 1 -- 172.21.15.7:0/781292674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00cdb0 con 0x7f7d1c0ad140 2024-09-15T23:37:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.810+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c1f6330 con 0x7f7d1c0ad140 2024-09-15T23:37:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.812+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1c06dea0 con 0x7f7d1c0ad140 2024-09-15T23:37:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.812+0000 7f7d11ffb640 1 -- 172.21.15.7:0/781292674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7d0c01a070 con 0x7f7d1c0ad140 2024-09-15T23:37:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.812+0000 7f7d11ffb640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7d0006c2b0 0x7f7d0006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.813+0000 7f7d11ffb640 1 -- 172.21.15.7:0/781292674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7d0c08b710 con 0x7f7d1c0ad140 2024-09-15T23:37:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.813+0000 7f7d21601640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7d0006c2b0 0x7f7d0006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.813+0000 7f7d21601640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7d0006c2b0 0x7f7d0006e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7d18034000 tx=0x7f7d1802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:01.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:01.817+0000 7f7d11ffb640 1 -- 172.21.15.7:0/781292674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d0c0593c0 con 0x7f7d1c0ad140 2024-09-15T23:37:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.007+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7d1c0af720 con 0x7f7d1c0ad140 2024-09-15T23:37:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.007+0000 7f7d11ffb640 1 -- 172.21.15.7:0/781292674 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f7d0c05d6c0 con 0x7f7d1c0ad140 2024-09-15T23:37:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-15T23:37:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.009+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7d0006c2b0 msgr2=0x7f7d0006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.009+0000 7f7d23865640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7d0006c2b0 0x7f7d0006e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7d18034000 tx=0x7f7d1802e000 comp rx=0 tx=0).stop 2024-09-15T23:37:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.009+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c0ad140 msgr2=0x7f7d1c0b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.009+0000 7f7d23865640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c0ad140 0x7f7d1c0b3f70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00ea00 tx=0x7f7d0c00eec0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.010+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 shutdown_connections 2024-09-15T23:37:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.010+0000 7f7d23865640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7d0006c2b0 0x7f7d0006e770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.010+0000 7f7d23865640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d1c159e30 0x7f7d1c14f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.010+0000 7f7d23865640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c0ad140 0x7f7d1c0b3f70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.010+0000 7f7d23865640 1 --2- 172.21.15.7:0/781292674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d1c06f520 0x7f7d1c1557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.010+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 >> 172.21.15.7:0/781292674 conn(0x7f7d1c0b1e60 msgr2=0x7f7d1c15b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.010+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 shutdown_connections 2024-09-15T23:37:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.010+0000 7f7d23865640 1 -- 172.21.15.7:0/781292674 wait complete. 2024-09-15T23:37:02.022 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-15T23:37:02.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:02.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-15T23:37:02.049 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:37:02.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:02.049 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-15T23:37:02.106 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:02.106 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:02.126 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:02.126 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:02.185 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:02.185 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:02.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-15T23:37:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.348+0000 7fee1e14b640 1 Processor -- start 2024-09-15T23:37:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.348+0000 7fee1e14b640 1 -- start start 2024-09-15T23:37:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee1e14b640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fee1809aa40 0x7fee1809ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee1e14b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee18070790 0x7fee18070c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee1e14b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee1809b360 con 0x7fee18160120 2024-09-15T23:37:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee1e14b640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fee18070300 con 0x7fee18070790 2024-09-15T23:37:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee1e14b640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fee180d1120 con 0x7fee1809aa40 2024-09-15T23:37:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee17fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee18070790 0x7fee18070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee16ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fee18160120 0x7fee180d06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56792/0 (socket says 172.21.15.7:56792) 2024-09-15T23:37:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee16ffd640 1 -- 172.21.15.7:0/69307190 learned_addr learned my addr 172.21.15.7:0/69307190 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee17fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee18070790 0x7fee18070c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60406/0 (socket says 172.21.15.7:60406) 2024-09-15T23:37:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee177fe640 1 --2- 172.21.15.7:0/69307190 >> v2:172.21.15.18:3300/0 conn(0x7fee1809aa40 0x7fee1809ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee17fff640 1 -- 172.21.15.7:0/69307190 >> v2:172.21.15.18:3300/0 conn(0x7fee1809aa40 msgr2=0x7fee1809ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee17fff640 1 --2- 172.21.15.7:0/69307190 >> v2:172.21.15.18:3300/0 conn(0x7fee1809aa40 0x7fee1809ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee17fff640 1 -- 172.21.15.7:0/69307190 >> v1:172.21.15.7:6789/0 conn(0x7fee18160120 legacy=0x7fee180d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee17fff640 1 -- 172.21.15.7:0/69307190 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee180d1400 con 0x7fee18070790 2024-09-15T23:37:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee167fc640 1 -- 172.21.15.7:0/69307190 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099621577 0 0) 0x7fee1809b360 con 0x7fee18160120 2024-09-15T23:37:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.349+0000 7fee177fe640 1 --2- 172.21.15.7:0/69307190 >> v2:172.21.15.18:3300/0 conn(0x7fee1809aa40 0x7fee1809ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.350+0000 7fee17fff640 1 --2- 172.21.15.7:0/69307190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee18070790 0x7fee18070c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee18069860 tx=0x7fee1002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20114bf89ee0c386 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.350+0000 7fee167fc640 1 -- 172.21.15.7:0/69307190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10007740 con 0x7fee18070790 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.350+0000 7fee167fc640 1 -- 172.21.15.7:0/69307190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee100078e0 con 0x7fee18070790 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.350+0000 7fee167fc640 1 -- 172.21.15.7:0/69307190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10007cb0 con 0x7fee18070790 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.351+0000 7fee1e14b640 1 -- 172.21.15.7:0/69307190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee18070790 msgr2=0x7fee18070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.351+0000 7fee1e14b640 1 --2- 172.21.15.7:0/69307190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee18070790 0x7fee18070c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee18069860 tx=0x7fee1002a680 comp rx=0 tx=0).stop 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.351+0000 7fee1e14b640 1 -- 172.21.15.7:0/69307190 shutdown_connections 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.351+0000 7fee1e14b640 1 --2- 172.21.15.7:0/69307190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee18070790 0x7fee18070c20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.351+0000 7fee1e14b640 1 --2- 172.21.15.7:0/69307190 >> v2:172.21.15.18:3300/0 conn(0x7fee1809aa40 0x7fee1809ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.351+0000 7fee1e14b640 1 -- 172.21.15.7:0/69307190 >> 172.21.15.7:0/69307190 conn(0x7fee180d2420 msgr2=0x7fee180d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.351+0000 7fee1e14b640 1 -- 172.21.15.7:0/69307190 shutdown_connections 2024-09-15T23:37:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.351+0000 7fee1e14b640 1 -- 172.21.15.7:0/69307190 wait complete. 2024-09-15T23:37:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.352+0000 7fee1e14b640 1 Processor -- start 2024-09-15T23:37:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.352+0000 7fee1e14b640 1 -- start start 2024-09-15T23:37:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee1e14b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fee18070790 0x7fee180ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee1e14b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1809aa40 0x7fee180b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee1e14b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee18160120 0x7fee181f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee1e14b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee180d0de0 con 0x7fee1809aa40 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee1e14b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fee180d19c0 con 0x7fee18160120 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee1e14b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fee18161f70 con 0x7fee18070790 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1809aa40 0x7fee180b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1809aa40 0x7fee180b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60418/0 (socket says 172.21.15.7:60418) 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee16ffd640 1 -- 172.21.15.7:0/1734121387 learned_addr learned my addr 172.21.15.7:0/1734121387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee16ffd640 1 -- 172.21.15.7:0/1734121387 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fee18070790 msgr2=0x7fee180ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee16ffd640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fee18070790 0x7fee180ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee16ffd640 1 -- 172.21.15.7:0/1734121387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee18160120 msgr2=0x7fee181f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee16ffd640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee18160120 0x7fee181f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.353+0000 7fee16ffd640 1 -- 172.21.15.7:0/1734121387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee180b1540 con 0x7fee1809aa40 2024-09-15T23:37:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.354+0000 7fee16ffd640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1809aa40 0x7fee180b1000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00c500 tx=0x7fee0c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.354+0000 7fedfffff640 1 -- 172.21.15.7:0/1734121387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c018070 con 0x7fee1809aa40 2024-09-15T23:37:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.354+0000 7fedfffff640 1 -- 172.21.15.7:0/1734121387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0c007580 con 0x7fee1809aa40 2024-09-15T23:37:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.354+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee180cab50 con 0x7fee1809aa40 2024-09-15T23:37:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.354+0000 7fedfffff640 1 -- 172.21.15.7:0/1734121387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c004a20 con 0x7fee1809aa40 2024-09-15T23:37:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.354+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee180cb110 con 0x7fee1809aa40 2024-09-15T23:37:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.355+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1809ae20 con 0x7fee1809aa40 2024-09-15T23:37:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.355+0000 7fedfffff640 1 -- 172.21.15.7:0/1734121387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fee0c01b070 con 0x7fee1809aa40 2024-09-15T23:37:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.356+0000 7fedfffff640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fee0406c2e0 0x7fee0406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.356+0000 7fedfffff640 1 -- 172.21.15.7:0/1734121387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fee0c08a9d0 con 0x7fee1809aa40 2024-09-15T23:37:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.356+0000 7fee177fe640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fee0406c2e0 0x7fee0406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.356+0000 7fee177fe640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fee0406c2e0 0x7fee0406e7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fee08006fd0 tx=0x7fee08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.360+0000 7fedfffff640 1 -- 172.21.15.7:0/1734121387 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fee0c058e70 con 0x7fee1809aa40 2024-09-15T23:37:02.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.547+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fee180d3250 con 0x7fee1809aa40 2024-09-15T23:37:02.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.548+0000 7fedfffff640 1 -- 172.21.15.7:0/1734121387 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fee0c058770 con 0x7fee1809aa40 2024-09-15T23:37:02.550 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-15T23:37:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fee0406c2e0 msgr2=0x7fee0406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fee0406c2e0 0x7fee0406e7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fee08006fd0 tx=0x7fee08008040 comp rx=0 tx=0).stop 2024-09-15T23:37:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1809aa40 msgr2=0x7fee180b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1809aa40 0x7fee180b1000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00c500 tx=0x7fee0c00c9c0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 shutdown_connections 2024-09-15T23:37:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee18160120 0x7fee181f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1809aa40 0x7fee180b1000 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fee0406c2e0 0x7fee0406e7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 --2- 172.21.15.7:0/1734121387 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fee18070790 0x7fee180ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 >> 172.21.15.7:0/1734121387 conn(0x7fee180d2420 msgr2=0x7fee180cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:02.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 shutdown_connections 2024-09-15T23:37:02.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.550+0000 7fee1e14b640 1 -- 172.21.15.7:0/1734121387 wait complete. 2024-09-15T23:37:02.563 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-15T23:37:02.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:02.563 DEBUG:teuthology.orchestra.run.smithi018:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-15T23:37:02.595 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:37:02.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:02.595 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-15T23:37:02.655 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-15T23:37:02.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:02.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.755+0000 7fe266b5f640 1 Processor -- start 2024-09-15T23:37:02.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.755+0000 7fe266b5f640 1 -- start start 2024-09-15T23:37:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.756+0000 7fe266b5f640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe2600b3b00 0x7fe2600cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.756+0000 7fe266b5f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26006fa40 con 0x7fe26015e860 2024-09-15T23:37:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.756+0000 7fe266b5f640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26006f1b0 con 0x7fe26015ec40 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.756+0000 7fe266b5f640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe2600b3f50 con 0x7fe2600b3b00 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe2648fb640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe26015e860 0x7fe2600cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56806/0 (socket says 172.21.15.7:56806) 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe2648fb640 1 -- 172.21.15.7:0/720308773 learned_addr learned my addr 172.21.15.7:0/720308773 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe2650fc640 1 --2- 172.21.15.7:0/720308773 >> v2:172.21.15.18:3300/0 conn(0x7fe2600b3b00 0x7fe2600cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 22589551 0 0) 0x7fe26006fa40 con 0x7fe26015e860 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe24c003520 con 0x7fe26015e860 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095977267 0 0) 0x7fe26006f1b0 con 0x7fe26015ec40 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe26006fa40 con 0x7fe26015ec40 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 431597241 0 0) 0x7fe24c003520 con 0x7fe26015e860 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 >> v2:172.21.15.18:3300/0 conn(0x7fe2600b3b00 msgr2=0x7fe2600cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 --2- 172.21.15.7:0/720308773 >> v2:172.21.15.18:3300/0 conn(0x7fe2600b3b00 0x7fe2600cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 >> v1:172.21.15.18:6789/0 conn(0x7fe26015ec40 legacy=0x7fe260162520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26006f490 con 0x7fe26015e860 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7fe250001690 con 0x7fe26015e860 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 >> v1:172.21.15.7:6789/0 conn(0x7fe26015e860 legacy=0x7fe2600cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe24c003b90 0x7fe24c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c004800 0x7fe24c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe24c005280 0x7fe24c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2600b3f50 con 0x7fe24c004800 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c00c1c0 con 0x7fe24c005280 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe24c00c3c0 con 0x7fe24c003b90 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe25ffff640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c004800 0x7fe24c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.757+0000 7fe2648fb640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe24c003b90 0x7fe24c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.758+0000 7fe2650fc640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe24c005280 0x7fe24c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.758+0000 7fe25ffff640 1 -- 172.21.15.7:0/720308773 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe24c003b90 msgr2=0x7fe24c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.758+0000 7fe25ffff640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe24c003b90 0x7fe24c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.758+0000 7fe25ffff640 1 -- 172.21.15.7:0/720308773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe24c005280 msgr2=0x7fe24c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.758+0000 7fe25ffff640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe24c005280 0x7fe24c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.758+0000 7fe2650fc640 1 -- 172.21.15.7:0/720308773 reap_dead start 2024-09-15T23:37:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.758+0000 7fe25ffff640 1 -- 172.21.15.7:0/720308773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c003520 con 0x7fe24c004800 2024-09-15T23:37:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.758+0000 7fe25ffff640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c004800 0x7fe24c004c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2480018d0 tx=0x7fe24c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d17ff5a109c33b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480014f0 con 0x7fe24c004800 2024-09-15T23:37:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2480052d0 con 0x7fe24c004800 2024-09-15T23:37:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe25f7fe640 1 -- 172.21.15.7:0/720308773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24802ac40 con 0x7fe24c004800 2024-09-15T23:37:02.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe266b5f640 1 -- 172.21.15.7:0/720308773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c004800 msgr2=0x7fe24c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe266b5f640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c004800 0x7fe24c004c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2480018d0 tx=0x7fe24c005830 comp rx=0 tx=0).stop 2024-09-15T23:37:02.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe266b5f640 1 -- 172.21.15.7:0/720308773 shutdown_connections 2024-09-15T23:37:02.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe266b5f640 1 --2- 172.21.15.7:0/720308773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c004800 0x7fe24c004c80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe266b5f640 1 -- 172.21.15.7:0/720308773 >> 172.21.15.7:0/720308773 conn(0x7fe2600b2fe0 msgr2=0x7fe2600b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe266b5f640 1 -- 172.21.15.7:0/720308773 shutdown_connections 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.759+0000 7fe266b5f640 1 -- 172.21.15.7:0/720308773 wait complete. 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.760+0000 7fe266b5f640 1 Processor -- start 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.760+0000 7fe266b5f640 1 -- start start 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.761+0000 7fe266b5f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe24c004800 0x7fe2601f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.761+0000 7fe266b5f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe26015f220 0x7fe2601f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.761+0000 7fe266b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2601f1660 0x7fe26015d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.761+0000 7fe266b5f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe26006ffc0 con 0x7fe2601f1660 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.761+0000 7fe266b5f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe2600cb6f0 con 0x7fe26015f220 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.761+0000 7fe266b5f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe260070140 con 0x7fe24c004800 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.761+0000 7fe2650fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2601f1660 0x7fe26015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.761+0000 7fe2650fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2601f1660 0x7fe26015d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60438/0 (socket says 172.21.15.7:60438) 2024-09-15T23:37:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe2650fc640 1 -- 172.21.15.7:0/2128936709 learned_addr learned my addr 172.21.15.7:0/2128936709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe25ffff640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe26015f220 0x7fe2601f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe2648fb640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe24c004800 0x7fe2601f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe2650fc640 1 -- 172.21.15.7:0/2128936709 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe24c004800 msgr2=0x7fe2601f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe2650fc640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe24c004800 0x7fe2601f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe2650fc640 1 -- 172.21.15.7:0/2128936709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe26015f220 msgr2=0x7fe2601f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe2650fc640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe26015f220 0x7fe2601f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe2650fc640 1 -- 172.21.15.7:0/2128936709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe260161ad0 con 0x7fe2601f1660 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe2650fc640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2601f1660 0x7fe26015d430 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe260162480 tx=0x7fe25800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe25d7fa640 1 -- 172.21.15.7:0/2128936709 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25800fde0 con 0x7fe2601f1660 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe25d7fa640 1 -- 172.21.15.7:0/2128936709 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25801d070 con 0x7fe2601f1660 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe260161db0 con 0x7fe2601f1660 2024-09-15T23:37:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe25d7fa640 1 -- 172.21.15.7:0/2128936709 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258015670 con 0x7fe2601f1660 2024-09-15T23:37:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.762+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2601f7ac0 con 0x7fe2601f1660 2024-09-15T23:37:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.763+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2601f78b0 con 0x7fe2601f1660 2024-09-15T23:37:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.764+0000 7fe25d7fa640 1 -- 172.21.15.7:0/2128936709 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe258015810 con 0x7fe2601f1660 2024-09-15T23:37:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.764+0000 7fe25d7fa640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe24006c2b0 0x7fe24006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.765+0000 7fe25d7fa640 1 -- 172.21.15.7:0/2128936709 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe25808c070 con 0x7fe2601f1660 2024-09-15T23:37:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.765+0000 7fe2648fb640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe24006c2b0 0x7fe24006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.765+0000 7fe2648fb640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe24006c2b0 0x7fe24006e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe2600cec50 tx=0x7fe250000f80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.769+0000 7fe25d7fa640 1 -- 172.21.15.7:0/2128936709 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe258010040 con 0x7fe2601f1660 2024-09-15T23:37:02.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.981+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2600ae2c0 con 0x7fe2601f1660 2024-09-15T23:37:02.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.982+0000 7fe25d7fa640 1 -- 172.21.15.7:0/2128936709 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5210 (secure 0 0 0) 0x7fe258059db0 con 0x7fe2601f1660 2024-09-15T23:37:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:02.984 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-15T23:36:42:878666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:42.233279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:37:02.984 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-15T23:37:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe24006c2b0 msgr2=0x7fe24006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe24006c2b0 0x7fe24006e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe2600cec50 tx=0x7fe250000f80 comp rx=0 tx=0).stop 2024-09-15T23:37:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2601f1660 msgr2=0x7fe26015d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2601f1660 0x7fe26015d430 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe260162480 tx=0x7fe25800f580 comp rx=0 tx=0).stop 2024-09-15T23:37:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 shutdown_connections 2024-09-15T23:37:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2601f1660 0x7fe26015d430 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe26015f220 0x7fe2601f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe24006c2b0 0x7fe24006e770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 --2- 172.21.15.7:0/2128936709 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe24c004800 0x7fe2601f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:02.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 >> 172.21.15.7:0/2128936709 conn(0x7fe2600b2fe0 msgr2=0x7fe2600b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:02.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 shutdown_connections 2024-09-15T23:37:02.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:02.984+0000 7fe266b5f640 1 -- 172.21.15.7:0/2128936709 wait complete. 2024-09-15T23:37:02.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:03.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.098+0000 7f4a217c6640 1 Processor -- start 2024-09-15T23:37:03.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.098+0000 7f4a217c6640 1 -- start start 2024-09-15T23:37:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.098+0000 7f4a217c6640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a1c0cd540 0x7f4a1c0b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.098+0000 7f4a217c6640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4a1c0b03e0 0x7f4a1c0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a217c6640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a1c06bf60 con 0x7f4a1c15e7a0 2024-09-15T23:37:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a217c6640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1c06c240 con 0x7f4a1c0cd540 2024-09-15T23:37:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a217c6640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1c06c520 con 0x7f4a1c0b03e0 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a1a7fc640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a1c0cd540 0x7f4a1c0b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a1a7fc640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a1c0cd540 0x7f4a1c0b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60444/0 (socket says 172.21.15.7:60444) 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a1a7fc640 1 -- 172.21.15.7:0/2101800683 learned_addr learned my addr 172.21.15.7:0/2101800683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a1b7fe640 1 --2- 172.21.15.7:0/2101800683 >> v2:172.21.15.18:3300/0 conn(0x7f4a1c0b03e0 0x7f4a1c0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a1a7fc640 1 -- 172.21.15.7:0/2101800683 >> v2:172.21.15.18:3300/0 conn(0x7f4a1c0b03e0 msgr2=0x7f4a1c0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a1a7fc640 1 --2- 172.21.15.7:0/2101800683 >> v2:172.21.15.18:3300/0 conn(0x7f4a1c0b03e0 0x7f4a1c0cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a1a7fc640 1 -- 172.21.15.7:0/2101800683 >> v1:172.21.15.18:6789/0 conn(0x7f4a1c15e7a0 legacy=0x7f4a1c0d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a1a7fc640 1 -- 172.21.15.7:0/2101800683 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c06c800 con 0x7f4a1c0cd540 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.099+0000 7f4a1b7fe640 1 --2- 172.21.15.7:0/2101800683 >> v2:172.21.15.18:3300/0 conn(0x7f4a1c0b03e0 0x7f4a1c0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.100+0000 7f4a1a7fc640 1 --2- 172.21.15.7:0/2101800683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 0x7f4a1c0b0930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c072b30 tx=0x7f4a1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8357f555f9fa3f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.100+0000 7f4a19ffb640 1 -- 172.21.15.7:0/2101800683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10037070 con 0x7f4a1c0cd540 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.100+0000 7f4a19ffb640 1 -- 172.21.15.7:0/2101800683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a10032440 con 0x7f4a1c0cd540 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.100+0000 7f4a19ffb640 1 -- 172.21.15.7:0/2101800683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10032810 con 0x7f4a1c0cd540 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.100+0000 7f4a217c6640 1 -- 172.21.15.7:0/2101800683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 msgr2=0x7f4a1c0b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.100+0000 7f4a217c6640 1 --2- 172.21.15.7:0/2101800683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 0x7f4a1c0b0930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c072b30 tx=0x7f4a1002a1a0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.101+0000 7f4a217c6640 1 -- 172.21.15.7:0/2101800683 shutdown_connections 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.101+0000 7f4a217c6640 1 --2- 172.21.15.7:0/2101800683 >> v2:172.21.15.18:3300/0 conn(0x7f4a1c0b03e0 0x7f4a1c0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.101+0000 7f4a217c6640 1 --2- 172.21.15.7:0/2101800683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 0x7f4a1c0b0930 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.101+0000 7f4a217c6640 1 -- 172.21.15.7:0/2101800683 >> 172.21.15.7:0/2101800683 conn(0x7f4a1c09aa40 msgr2=0x7f4a1c0b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.101+0000 7f4a217c6640 1 -- 172.21.15.7:0/2101800683 shutdown_connections 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.101+0000 7f4a217c6640 1 -- 172.21.15.7:0/2101800683 wait complete. 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.102+0000 7f4a217c6640 1 Processor -- start 2024-09-15T23:37:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.102+0000 7f4a217c6640 1 -- start start 2024-09-15T23:37:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a217c6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c0b03e0 0x7f4a1c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a217c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 0x7f4a1c1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a217c6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a1c15e7a0 0x7f4a1c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a217c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c0d0d90 con 0x7f4a1c0cd540 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a217c6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c15eb80 con 0x7f4a1c0b03e0 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a217c6640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c0d0f10 con 0x7f4a1c15e7a0 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 0x7f4a1c1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 0x7f4a1c1567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60452/0 (socket says 172.21.15.7:60452) 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1a7fc640 1 -- 172.21.15.7:0/1576948709 learned_addr learned my addr 172.21.15.7:0/1576948709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1b7fe640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a1c15e7a0 0x7f4a1c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1affd640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c0b03e0 0x7f4a1c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1a7fc640 1 -- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a1c15e7a0 msgr2=0x7f4a1c157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1a7fc640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a1c15e7a0 0x7f4a1c157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1a7fc640 1 -- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c0b03e0 msgr2=0x7f4a1c0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1a7fc640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c0b03e0 0x7f4a1c0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1a7fc640 1 -- 172.21.15.7:0/1576948709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c0c7fa0 con 0x7f4a1c0cd540 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1b7fe640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a1c15e7a0 0x7f4a1c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1affd640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c0b03e0 0x7f4a1c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.103+0000 7f4a1a7fc640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 0x7f4a1c1567a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a1002a1e0 tx=0x7f4a100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.104+0000 7f4a037fe640 1 -- 172.21.15.7:0/1576948709 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10037040 con 0x7f4a1c0cd540 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.104+0000 7f4a037fe640 1 -- 172.21.15.7:0/1576948709 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a10042070 con 0x7f4a1c0cd540 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.104+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c0c4e00 con 0x7f4a1c0cd540 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.104+0000 7f4a037fe640 1 -- 172.21.15.7:0/1576948709 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10030bb0 con 0x7f4a1c0cd540 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.104+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c0c53c0 con 0x7f4a1c0cd540 2024-09-15T23:37:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.105+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1c0b0930 con 0x7f4a1c0cd540 2024-09-15T23:37:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.105+0000 7f4a037fe640 1 -- 172.21.15.7:0/1576948709 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4a10047460 con 0x7f4a1c0cd540 2024-09-15T23:37:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.106+0000 7f4a037fe640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4a0806c2b0 0x7f4a0806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.106+0000 7f4a037fe640 1 -- 172.21.15.7:0/1576948709 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4a100ac440 con 0x7f4a1c0cd540 2024-09-15T23:37:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.106+0000 7f4a1affd640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4a0806c2b0 0x7f4a0806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.106+0000 7f4a1affd640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4a0806c2b0 0x7f4a0806e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c003eb0 tx=0x7f4a0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.110+0000 7f4a037fe640 1 -- 172.21.15.7:0/1576948709 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a10045360 con 0x7f4a1c0cd540 2024-09-15T23:37:03.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.323+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a1c162d80 con 0x7f4a1c0cd540 2024-09-15T23:37:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.323+0000 7f4a037fe640 1 -- 172.21.15.7:0/1576948709 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5210 (secure 0 0 0) 0x7f4a1007a7d0 con 0x7f4a1c0cd540 2024-09-15T23:37:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:03.325 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-15T23:36:42:878666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:42.233279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:37:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-15T23:37:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4a0806c2b0 msgr2=0x7f4a0806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4a0806c2b0 0x7f4a0806e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c003eb0 tx=0x7f4a0c009040 comp rx=0 tx=0).stop 2024-09-15T23:37:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 msgr2=0x7f4a1c1567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 0x7f4a1c1567a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a1002a1e0 tx=0x7f4a100046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 shutdown_connections 2024-09-15T23:37:03.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f4a0806c2b0 0x7f4a0806e770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a1c15e7a0 0x7f4a1c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1c0cd540 0x7f4a1c1567a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 --2- 172.21.15.7:0/1576948709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c0b03e0 0x7f4a1c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 >> 172.21.15.7:0/1576948709 conn(0x7f4a1c09aa40 msgr2=0x7f4a1c0b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:03.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 shutdown_connections 2024-09-15T23:37:03.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.325+0000 7f4a217c6640 1 -- 172.21.15.7:0/1576948709 wait complete. 2024-09-15T23:37:03.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:37:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.438+0000 7fbcf7c17640 1 Processor -- start 2024-09-15T23:37:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.438+0000 7fbcf7c17640 1 -- start start 2024-09-15T23:37:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.439+0000 7fbcf7c17640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf009b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.439+0000 7fbcf7c17640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbcf006ca60 0x7fbcf006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.439+0000 7fbcf7c17640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf00d14a0 con 0x7fbcf00d0c50 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.439+0000 7fbcf7c17640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf015ec50 con 0x7fbcf006ca60 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.439+0000 7fbcf7c17640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf00d1780 con 0x7fbcf015e870 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.439+0000 7fbcf59b3640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf009b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.439+0000 7fbcf61b4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbcf006ca60 0x7fbcf006cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.439+0000 7fbcf59b3640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf009b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:53840/0 (socket says 172.21.15.7:53840) 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.439+0000 7fbcf59b3640 1 -- 172.21.15.7:0/3712875763 learned_addr learned my addr 172.21.15.7:0/3712875763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.440+0000 7fbcf59b3640 1 -- 172.21.15.7:0/3712875763 >> v2:172.21.15.18:3300/0 conn(0x7fbcf006ca60 msgr2=0x7fbcf006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.440+0000 7fbcf59b3640 1 --2- 172.21.15.7:0/3712875763 >> v2:172.21.15.18:3300/0 conn(0x7fbcf006ca60 0x7fbcf006cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.440+0000 7fbcf59b3640 1 -- 172.21.15.7:0/3712875763 >> v1:172.21.15.18:6789/0 conn(0x7fbcf00d0c50 legacy=0x7fbcf00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.440+0000 7fbcf59b3640 1 -- 172.21.15.7:0/3712875763 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf00cd850 con 0x7fbcf015e870 2024-09-15T23:37:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf59b3640 1 --2- 172.21.15.7:0/3712875763 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf009b610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009870 tx=0x7fbce002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4653154bd7eb190e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf49b1640 1 -- 172.21.15.7:0/3712875763 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0037070 con 0x7fbcf015e870 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf49b1640 1 -- 172.21.15.7:0/3712875763 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce002ae50 con 0x7fbcf015e870 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf49b1640 1 -- 172.21.15.7:0/3712875763 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0032860 con 0x7fbcf015e870 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf7c17640 1 -- 172.21.15.7:0/3712875763 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 msgr2=0x7fbcf009b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf7c17640 1 --2- 172.21.15.7:0/3712875763 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf009b610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009870 tx=0x7fbce002a300 comp rx=0 tx=0).stop 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf7c17640 1 -- 172.21.15.7:0/3712875763 shutdown_connections 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf7c17640 1 --2- 172.21.15.7:0/3712875763 >> v2:172.21.15.18:3300/0 conn(0x7fbcf006ca60 0x7fbcf006cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf7c17640 1 --2- 172.21.15.7:0/3712875763 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf009b610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf7c17640 1 -- 172.21.15.7:0/3712875763 >> 172.21.15.7:0/3712875763 conn(0x7fbcf00b3f30 msgr2=0x7fbcf00b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf7c17640 1 -- 172.21.15.7:0/3712875763 shutdown_connections 2024-09-15T23:37:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.441+0000 7fbcf7c17640 1 -- 172.21.15.7:0/3712875763 wait complete. 2024-09-15T23:37:03.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.442+0000 7fbcf7c17640 1 Processor -- start 2024-09-15T23:37:03.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf7c17640 1 -- start start 2024-09-15T23:37:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf7c17640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf006ca60 0x7fbcf0159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf7c17640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf00d0c50 0x7fbcf01f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf7c17640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf00b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf7c17640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf00ce760 con 0x7fbcf006ca60 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf7c17640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf00cd550 con 0x7fbcf00d0c50 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf7c17640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf00d1070 con 0x7fbcf015e870 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf59b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf006ca60 0x7fbcf0159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf59b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf006ca60 0x7fbcf0159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60456/0 (socket says 172.21.15.7:60456) 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.443+0000 7fbcf59b3640 1 -- 172.21.15.7:0/4042654239 learned_addr learned my addr 172.21.15.7:0/4042654239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf51b2640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf00d0c50 0x7fbcf01f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf61b4640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf00b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf59b3640 1 -- 172.21.15.7:0/4042654239 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 msgr2=0x7fbcf00b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf59b3640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf00b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf59b3640 1 -- 172.21.15.7:0/4042654239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf00d0c50 msgr2=0x7fbcf01f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf59b3640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf00d0c50 0x7fbcf01f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf59b3640 1 -- 172.21.15.7:0/4042654239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf015a2b0 con 0x7fbcf006ca60 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf59b3640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf006ca60 0x7fbcf0159d70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbce00090b0 tx=0x7fbce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbce67fc640 1 -- 172.21.15.7:0/4042654239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0042070 con 0x7fbcf006ca60 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf01602f0 con 0x7fbcf006ca60 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbce67fc640 1 -- 172.21.15.7:0/4042654239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce003e680 con 0x7fbcf006ca60 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbce67fc640 1 -- 172.21.15.7:0/4042654239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0037040 con 0x7fbcf006ca60 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.444+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf01f54f0 con 0x7fbcf006ca60 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.445+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf009b610 con 0x7fbcf006ca60 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.446+0000 7fbce67fc640 1 -- 172.21.15.7:0/4042654239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbce0045070 con 0x7fbcf006ca60 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.446+0000 7fbce67fc640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcd406c2b0 0x7fbcd406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.446+0000 7fbce67fc640 1 -- 172.21.15.7:0/4042654239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbce00abfd0 con 0x7fbcf006ca60 2024-09-15T23:37:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.446+0000 7fbcf51b2640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcd406c2b0 0x7fbcd406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.447+0000 7fbcf51b2640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcd406c2b0 0x7fbcd406e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbcf01f7ea0 tx=0x7fbcec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.451+0000 7fbce67fc640 1 -- 172.21.15.7:0/4042654239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbce007a410 con 0x7fbcf006ca60 2024-09-15T23:37:03.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.629+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcf01f5890 con 0x7fbcf006ca60 2024-09-15T23:37:03.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.630+0000 7fbce67fc640 1 -- 172.21.15.7:0/4042654239 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18506 (secure 0 0 0) 0x7fbce0079d10 con 0x7fbcf006ca60 2024-09-15T23:37:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:03.633 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:36:27.958299+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:37:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcd406c2b0 msgr2=0x7fbcd406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcd406c2b0 0x7fbcd406e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbcf01f7ea0 tx=0x7fbcec009040 comp rx=0 tx=0).stop 2024-09-15T23:37:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf006ca60 msgr2=0x7fbcf0159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf006ca60 0x7fbcf0159d70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbce00090b0 tx=0x7fbce00046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 shutdown_connections 2024-09-15T23:37:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcd406c2b0 0x7fbcd406e770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf015e870 0x7fbcf00b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf00d0c50 0x7fbcf01f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 --2- 172.21.15.7:0/4042654239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf006ca60 0x7fbcf0159d70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 >> 172.21.15.7:0/4042654239 conn(0x7fbcf00b3f30 msgr2=0x7fbcf0071840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 shutdown_connections 2024-09-15T23:37:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.632+0000 7fbcf7c17640 1 -- 172.21.15.7:0/4042654239 wait complete. 2024-09-15T23:37:03.645 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-15T23:37:03.645 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.745+0000 7fe4b8cd0640 1 Processor -- start 2024-09-15T23:37:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.745+0000 7fe4b8cd0640 1 -- start start 2024-09-15T23:37:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.746+0000 7fe4b8cd0640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4b415e840 0x7fe4b40d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.746+0000 7fe4b8cd0640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe4b40d52e0 0x7fe4b406d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.746+0000 7fe4b8cd0640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b40cf560 con 0x7fe4b406dbb0 2024-09-15T23:37:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.746+0000 7fe4b8cd0640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b40cfe90 con 0x7fe4b415e840 2024-09-15T23:37:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.746+0000 7fe4b8cd0640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b40cf840 con 0x7fe4b40d52e0 2024-09-15T23:37:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.746+0000 7fe4b259c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4b415e840 0x7fe4b40d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.746+0000 7fe4b259c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4b415e840 0x7fe4b40d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60458/0 (socket says 172.21.15.7:60458) 2024-09-15T23:37:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.746+0000 7fe4b259c640 1 -- 172.21.15.7:0/2784344776 learned_addr learned my addr 172.21.15.7:0/2784344776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.747+0000 7fe4b1d9b640 1 --2- 172.21.15.7:0/2784344776 >> v2:172.21.15.18:3300/0 conn(0x7fe4b40d52e0 0x7fe4b406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.747+0000 7fe4b259c640 1 -- 172.21.15.7:0/2784344776 >> v2:172.21.15.18:3300/0 conn(0x7fe4b40d52e0 msgr2=0x7fe4b406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.747+0000 7fe4b259c640 1 --2- 172.21.15.7:0/2784344776 >> v2:172.21.15.18:3300/0 conn(0x7fe4b40d52e0 0x7fe4b406d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.747+0000 7fe4b259c640 1 -- 172.21.15.7:0/2784344776 >> v1:172.21.15.18:6789/0 conn(0x7fe4b406dbb0 legacy=0x7fe4b40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.747+0000 7fe4b259c640 1 -- 172.21.15.7:0/2784344776 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b40cfb20 con 0x7fe4b415e840 2024-09-15T23:37:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.747+0000 7fe4b259c640 1 --2- 172.21.15.7:0/2784344776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b415e840 0x7fe4b40d0320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40c7e60 tx=0x7fe4ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2800ed5c0c9b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b159a640 1 -- 172.21.15.7:0/2784344776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac037070 con 0x7fe4b415e840 2024-09-15T23:37:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b159a640 1 -- 172.21.15.7:0/2784344776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac032440 con 0x7fe4b415e840 2024-09-15T23:37:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b159a640 1 -- 172.21.15.7:0/2784344776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac032810 con 0x7fe4b415e840 2024-09-15T23:37:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/2784344776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b415e840 msgr2=0x7fe4b40d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b8cd0640 1 --2- 172.21.15.7:0/2784344776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b415e840 0x7fe4b40d0320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40c7e60 tx=0x7fe4ac02a230 comp rx=0 tx=0).stop 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/2784344776 shutdown_connections 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b8cd0640 1 --2- 172.21.15.7:0/2784344776 >> v2:172.21.15.18:3300/0 conn(0x7fe4b40d52e0 0x7fe4b406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b8cd0640 1 --2- 172.21.15.7:0/2784344776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b415e840 0x7fe4b40d0320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/2784344776 >> 172.21.15.7:0/2784344776 conn(0x7fe4b408bfd0 msgr2=0x7fe4b40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/2784344776 shutdown_connections 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.748+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/2784344776 wait complete. 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.750+0000 7fe4b8cd0640 1 Processor -- start 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.750+0000 7fe4b8cd0640 1 -- start start 2024-09-15T23:37:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.750+0000 7fe4b8cd0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b406dbb0 0x7fe4b4151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.750+0000 7fe4b8cd0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4b40d52e0 0x7fe4b40cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.750+0000 7fe4b8cd0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b415e840 0x7fe4b40c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.750+0000 7fe4b8cd0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b415ec20 con 0x7fe4b406dbb0 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.750+0000 7fe4b8cd0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b40d0860 con 0x7fe4b415e840 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.750+0000 7fe4b8cd0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b40cd7d0 con 0x7fe4b40d52e0 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.750+0000 7fe4b259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b406dbb0 0x7fe4b4151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b406dbb0 0x7fe4b4151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60460/0 (socket says 172.21.15.7:60460) 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b259c640 1 -- 172.21.15.7:0/1082919648 learned_addr learned my addr 172.21.15.7:0/1082919648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b1d9b640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4b40d52e0 0x7fe4b40cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b2d9d640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b415e840 0x7fe4b40c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b259c640 1 -- 172.21.15.7:0/1082919648 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4b40d52e0 msgr2=0x7fe4b40cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b259c640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4b40d52e0 0x7fe4b40cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b259c640 1 -- 172.21.15.7:0/1082919648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b415e840 msgr2=0x7fe4b40c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b259c640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b415e840 0x7fe4b40c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b259c640 1 -- 172.21.15.7:0/1082919648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b40cdab0 con 0x7fe4b406dbb0 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b259c640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b406dbb0 0x7fe4b4151b60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac000c00 tx=0x7fe4ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4aaffd640 1 -- 172.21.15.7:0/1082919648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac041020 con 0x7fe4b406dbb0 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4aaffd640 1 -- 172.21.15.7:0/1082919648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac030970 con 0x7fe4b406dbb0 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4aaffd640 1 -- 172.21.15.7:0/1082919648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac037040 con 0x7fe4b406dbb0 2024-09-15T23:37:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b40cdd90 con 0x7fe4b406dbb0 2024-09-15T23:37:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.751+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b40b5110 con 0x7fe4b406dbb0 2024-09-15T23:37:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.752+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b406d670 con 0x7fe4b406dbb0 2024-09-15T23:37:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.753+0000 7fe4aaffd640 1 -- 172.21.15.7:0/1082919648 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe4ac044020 con 0x7fe4b406dbb0 2024-09-15T23:37:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.753+0000 7fe4aaffd640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe49406c2b0 0x7fe49406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.753+0000 7fe4aaffd640 1 -- 172.21.15.7:0/1082919648 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe4ac0ac1c0 con 0x7fe4b406dbb0 2024-09-15T23:37:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.753+0000 7fe4b1d9b640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe49406c2b0 0x7fe49406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.754+0000 7fe4b1d9b640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe49406c2b0 0x7fe49406e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40b6c70 tx=0x7fe49c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:03.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.758+0000 7fe4aaffd640 1 -- 172.21.15.7:0/1082919648 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe4ac04b030 con 0x7fe4b406dbb0 2024-09-15T23:37:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.975+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4b40d1cb0 con 0x7fe4b406dbb0 2024-09-15T23:37:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.976+0000 7fe4aaffd640 1 -- 172.21.15.7:0/1082919648 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5210 (secure 0 0 0) 0x7fe4ac07a560 con 0x7fe4b406dbb0 2024-09-15T23:37:03.978 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:03.978 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-15T23:36:42:878666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:42.233279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:37:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-15T23:37:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe49406c2b0 msgr2=0x7fe49406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe49406c2b0 0x7fe49406e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40b6c70 tx=0x7fe49c008040 comp rx=0 tx=0).stop 2024-09-15T23:37:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b406dbb0 msgr2=0x7fe4b4151b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b406dbb0 0x7fe4b4151b60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac000c00 tx=0x7fe4ac0046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 shutdown_connections 2024-09-15T23:37:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b415e840 0x7fe4b40c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe49406c2b0 0x7fe49406e770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4b40d52e0 0x7fe4b40cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 --2- 172.21.15.7:0/1082919648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b406dbb0 0x7fe4b4151b60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 >> 172.21.15.7:0/1082919648 conn(0x7fe4b408bfd0 msgr2=0x7fe4b40b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 shutdown_connections 2024-09-15T23:37:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:03.978+0000 7fe4b8cd0640 1 -- 172.21.15.7:0/1082919648 wait complete. 2024-09-15T23:37:03.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.091+0000 7f98a29b9640 1 Processor -- start 2024-09-15T23:37:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.091+0000 7f98a29b9640 1 -- start start 2024-09-15T23:37:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.092+0000 7f98a29b9640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f989c0d32a0 0x7f989c169280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.092+0000 7f98a29b9640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f989c0b3d10 con 0x7f989c0d51c0 2024-09-15T23:37:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.092+0000 7f98a29b9640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f989c0cfcd0 con 0x7f989c15e6d0 2024-09-15T23:37:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.092+0000 7f98a29b9640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f989c0d28c0 con 0x7f989c0d32a0 2024-09-15T23:37:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.092+0000 7f98a0f56640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f989c0d32a0 0x7f989c169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.092+0000 7f989b7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f989c0d51c0 0x7f989c0d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56814/0 (socket says 172.21.15.7:56814) 2024-09-15T23:37:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.092+0000 7f989b7fe640 1 -- 172.21.15.7:0/1063390256 learned_addr learned my addr 172.21.15.7:0/1063390256 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.093+0000 7f98a0f56640 1 -- 172.21.15.7:0/1063390256 >> v1:172.21.15.18:6789/0 conn(0x7f989c15e6d0 legacy=0x7f989c0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.093+0000 7f98a0f56640 1 -- 172.21.15.7:0/1063390256 >> v1:172.21.15.7:6789/0 conn(0x7f989c0d51c0 legacy=0x7f989c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.093+0000 7f98a0f56640 1 -- 172.21.15.7:0/1063390256 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989c0cff50 con 0x7f989c0d32a0 2024-09-15T23:37:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.093+0000 7f98a0f56640 1 --2- 172.21.15.7:0/1063390256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0d32a0 0x7f989c169280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9894009d90 tx=0x7f989402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1776da1c9a207ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.093+0000 7f989affd640 1 -- 172.21.15.7:0/1063390256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894007740 con 0x7f989c0d32a0 2024-09-15T23:37:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.094+0000 7f989affd640 1 -- 172.21.15.7:0/1063390256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98940078e0 con 0x7f989c0d32a0 2024-09-15T23:37:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.094+0000 7f989affd640 1 -- 172.21.15.7:0/1063390256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894007cb0 con 0x7f989c0d32a0 2024-09-15T23:37:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.094+0000 7f98a29b9640 1 -- 172.21.15.7:0/1063390256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0d32a0 msgr2=0x7f989c169280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.094+0000 7f98a29b9640 1 --2- 172.21.15.7:0/1063390256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0d32a0 0x7f989c169280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9894009d90 tx=0x7f989402a6f0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.094+0000 7f98a29b9640 1 -- 172.21.15.7:0/1063390256 shutdown_connections 2024-09-15T23:37:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.094+0000 7f98a29b9640 1 --2- 172.21.15.7:0/1063390256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0d32a0 0x7f989c169280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.094+0000 7f98a29b9640 1 -- 172.21.15.7:0/1063390256 >> 172.21.15.7:0/1063390256 conn(0x7f989c08bfd0 msgr2=0x7f989c0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.094+0000 7f98a29b9640 1 -- 172.21.15.7:0/1063390256 shutdown_connections 2024-09-15T23:37:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.094+0000 7f98a29b9640 1 -- 172.21.15.7:0/1063390256 wait complete. 2024-09-15T23:37:04.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.095+0000 7f98a29b9640 1 Processor -- start 2024-09-15T23:37:04.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.095+0000 7f98a29b9640 1 -- start start 2024-09-15T23:37:04.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.096+0000 7f98a29b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f989c0d32a0 0x7f989c1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.096+0000 7f98a29b9640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f989c0d51c0 0x7f989c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.096+0000 7f98a29b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c15e6d0 0x7f989c0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.096+0000 7f98a29b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f989c0d0510 con 0x7f989c15e6d0 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.096+0000 7f98a29b9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f989c15eab0 con 0x7f989c0d32a0 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.096+0000 7f98a29b9640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f989c0b51d0 con 0x7f989c0d51c0 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.096+0000 7f98a0f56640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c15e6d0 0x7f989c0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.096+0000 7f98a0f56640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c15e6d0 0x7f989c0cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60486/0 (socket says 172.21.15.7:60486) 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.096+0000 7f98a0f56640 1 -- 172.21.15.7:0/3585994968 learned_addr learned my addr 172.21.15.7:0/3585994968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f989b7fe640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f989c0d51c0 0x7f989c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f989bfff640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f989c0d32a0 0x7f989c1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f98a0f56640 1 -- 172.21.15.7:0/3585994968 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f989c0d51c0 msgr2=0x7f989c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f98a0f56640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f989c0d51c0 0x7f989c0cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f98a0f56640 1 -- 172.21.15.7:0/3585994968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f989c0d32a0 msgr2=0x7f989c1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f98a0f56640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f989c0d32a0 0x7f989c1539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f98a0f56640 1 -- 172.21.15.7:0/3585994968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989c0cc2c0 con 0x7f989c15e6d0 2024-09-15T23:37:04.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f98a0f56640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c15e6d0 0x7f989c0cddc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9894009eb0 tx=0x7f9894004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:04.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f9898ff9640 1 -- 172.21.15.7:0/3585994968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894040070 con 0x7f989c15e6d0 2024-09-15T23:37:04.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f9898ff9640 1 -- 172.21.15.7:0/3585994968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9894031c90 con 0x7f989c15e6d0 2024-09-15T23:37:04.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989c0cc5a0 con 0x7f989c15e6d0 2024-09-15T23:37:04.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f9898ff9640 1 -- 172.21.15.7:0/3585994968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894045680 con 0x7f989c15e6d0 2024-09-15T23:37:04.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.097+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989c0ccb60 con 0x7f989c15e6d0 2024-09-15T23:37:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.098+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f989c0d2b90 con 0x7f989c15e6d0 2024-09-15T23:37:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.099+0000 7f9898ff9640 1 -- 172.21.15.7:0/3585994968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9894039040 con 0x7f989c15e6d0 2024-09-15T23:37:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.099+0000 7f9898ff9640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f988806c2b0 0x7f988806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.099+0000 7f9898ff9640 1 -- 172.21.15.7:0/3585994968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6590+0+0 (secure 0 0 0) 0x7f98940ad230 con 0x7f989c15e6d0 2024-09-15T23:37:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.099+0000 7f989bfff640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f988806c2b0 0x7f988806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.100+0000 7f989bfff640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f988806c2b0 0x7f988806e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f988c007fd0 tx=0x7f988c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.103+0000 7f9898ff9640 1 -- 172.21.15.7:0/3585994968 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9894043360 con 0x7f989c15e6d0 2024-09-15T23:37:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.314+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f989c0008d0 con 0x7f989c15e6d0 2024-09-15T23:37:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.315+0000 7f9898ff9640 1 -- 172.21.15.7:0/3585994968 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5210 (secure 0 0 0) 0x7f989407b660 con 0x7f989c15e6d0 2024-09-15T23:37:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:04.317 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-15T23:36:42:878666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/4163991262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":4163991262},{"type":"v1","addr":"172.21.15.7:6837","nonce":4163991262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/941990273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":941990273},{"type":"v1","addr":"172.21.15.7:6836","nonce":941990273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/2608438268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":2608438268},{"type":"v1","addr":"172.21.15.18:6835","nonce":2608438268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:36:22.916234+0000","modified":"2024-09-15T23:36:42.233279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3035612480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3035612480},{"type":"v1","addr":"172.21.15.18:6833","nonce":3035612480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-09-15T23:37:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-15T23:37:04.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f988806c2b0 msgr2=0x7f988806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f988806c2b0 0x7f988806e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f988c007fd0 tx=0x7f988c009040 comp rx=0 tx=0).stop 2024-09-15T23:37:04.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c15e6d0 msgr2=0x7f989c0cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c15e6d0 0x7f989c0cddc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9894009eb0 tx=0x7f9894004a40 comp rx=0 tx=0).stop 2024-09-15T23:37:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 shutdown_connections 2024-09-15T23:37:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c15e6d0 0x7f989c0cddc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f988806c2b0 0x7f988806e770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f989c0d51c0 0x7f989c0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 --2- 172.21.15.7:0/3585994968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f989c0d32a0 0x7f989c1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 >> 172.21.15.7:0/3585994968 conn(0x7f989c08bfd0 msgr2=0x7f989c0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 shutdown_connections 2024-09-15T23:37:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.317+0000 7f98a29b9640 1 -- 172.21.15.7:0/3585994968 wait complete. 2024-09-15T23:37:04.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:37:04.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.431+0000 7f2e8d766640 1 Processor -- start 2024-09-15T23:37:04.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.431+0000 7f2e8d766640 1 -- start start 2024-09-15T23:37:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.432+0000 7f2e8d766640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e8815e800 0x7f2e880b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.432+0000 7f2e8d766640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e880ae250 0x7f2e880b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.432+0000 7f2e8d766640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2e880b29d0 0x7f2e880b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.432+0000 7f2e8d766640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2e880b6030 con 0x7f2e880ae250 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.432+0000 7f2e8d766640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f2e880b67e0 con 0x7f2e880b29d0 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.432+0000 7f2e8d766640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f2e880b1ac0 con 0x7f2e8815e800 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.432+0000 7f2e867fc640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e880ae250 0x7f2e880b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.432+0000 7f2e867fc640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e880ae250 0x7f2e880b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60500/0 (socket says 172.21.15.7:60500) 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.432+0000 7f2e867fc640 1 -- 172.21.15.7:0/743515446 learned_addr learned my addr 172.21.15.7:0/743515446 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e877fe640 1 --2- 172.21.15.7:0/743515446 >> v2:172.21.15.18:3300/0 conn(0x7f2e880b29d0 0x7f2e880b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e86ffd640 1 --2- 172.21.15.7:0/743515446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e8815e800 0x7f2e880b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e867fc640 1 -- 172.21.15.7:0/743515446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e8815e800 msgr2=0x7f2e880b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e867fc640 1 --2- 172.21.15.7:0/743515446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e8815e800 0x7f2e880b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e867fc640 1 -- 172.21.15.7:0/743515446 >> v2:172.21.15.18:3300/0 conn(0x7f2e880b29d0 msgr2=0x7f2e880b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e867fc640 1 --2- 172.21.15.7:0/743515446 >> v2:172.21.15.18:3300/0 conn(0x7f2e880b29d0 0x7f2e880b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e867fc640 1 -- 172.21.15.7:0/743515446 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8809ae70 con 0x7f2e880ae250 2024-09-15T23:37:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e86ffd640 1 --2- 172.21.15.7:0/743515446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e8815e800 0x7f2e880b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e877fe640 1 --2- 172.21.15.7:0/743515446 >> v2:172.21.15.18:3300/0 conn(0x7f2e880b29d0 0x7f2e880b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.433+0000 7f2e867fc640 1 --2- 172.21.15.7:0/743515446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ae250 0x7f2e880b2490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e880d12b0 tx=0x7f2e7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b32c6e4187673e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e85ffb640 1 -- 172.21.15.7:0/743515446 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70037070 con 0x7f2e880ae250 2024-09-15T23:37:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e85ffb640 1 -- 172.21.15.7:0/743515446 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e70032440 con 0x7f2e880ae250 2024-09-15T23:37:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e85ffb640 1 -- 172.21.15.7:0/743515446 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70032810 con 0x7f2e880ae250 2024-09-15T23:37:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e8d766640 1 -- 172.21.15.7:0/743515446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ae250 msgr2=0x7f2e880b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e8d766640 1 --2- 172.21.15.7:0/743515446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ae250 0x7f2e880b2490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e880d12b0 tx=0x7f2e7002a1e0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e8d766640 1 -- 172.21.15.7:0/743515446 shutdown_connections 2024-09-15T23:37:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e8d766640 1 --2- 172.21.15.7:0/743515446 >> v2:172.21.15.18:3300/0 conn(0x7f2e880b29d0 0x7f2e880b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e8d766640 1 --2- 172.21.15.7:0/743515446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ae250 0x7f2e880b2490 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e8d766640 1 --2- 172.21.15.7:0/743515446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e8815e800 0x7f2e880b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e8d766640 1 -- 172.21.15.7:0/743515446 >> 172.21.15.7:0/743515446 conn(0x7f2e88073490 msgr2=0x7f2e880b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e8d766640 1 -- 172.21.15.7:0/743515446 shutdown_connections 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.434+0000 7f2e8d766640 1 -- 172.21.15.7:0/743515446 wait complete. 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e8d766640 1 Processor -- start 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e8d766640 1 -- start start 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e8d766640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e880ae250 0x7f2e88151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e8d766640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880b29d0 0x7f2e8809d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e8d766640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e8815e800 0x7f2e8806bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e8d766640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e880b0560 con 0x7f2e880b29d0 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e8d766640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2e880b6260 con 0x7f2e8815e800 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e8d766640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f2e880b06e0 con 0x7f2e880ae250 2024-09-15T23:37:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880b29d0 0x7f2e8809d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880b29d0 0x7f2e8809d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60512/0 (socket says 172.21.15.7:60512) 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.436+0000 7f2e867fc640 1 -- 172.21.15.7:0/1284585844 learned_addr learned my addr 172.21.15.7:0/1284585844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e877fe640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e8815e800 0x7f2e8806bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e86ffd640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e880ae250 0x7f2e88151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e867fc640 1 -- 172.21.15.7:0/1284585844 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e880ae250 msgr2=0x7f2e88151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e867fc640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e880ae250 0x7f2e88151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e867fc640 1 -- 172.21.15.7:0/1284585844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e8815e800 msgr2=0x7f2e8806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e867fc640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e8815e800 0x7f2e8806bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e867fc640 1 -- 172.21.15.7:0/1284585844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e881f4880 con 0x7f2e880b29d0 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e867fc640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880b29d0 0x7f2e8809d1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2e70000c00 tx=0x7f2e700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e7f7fe640 1 -- 172.21.15.7:0/1284585844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70037040 con 0x7f2e880b29d0 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e7f7fe640 1 -- 172.21.15.7:0/1284585844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e70042070 con 0x7f2e880b29d0 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e7f7fe640 1 -- 172.21.15.7:0/1284585844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70030bb0 con 0x7f2e880b29d0 2024-09-15T23:37:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e881f4b60 con 0x7f2e880b29d0 2024-09-15T23:37:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.437+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881f5120 con 0x7f2e880b29d0 2024-09-15T23:37:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.438+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8815ebe0 con 0x7f2e880b29d0 2024-09-15T23:37:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.439+0000 7f2e7f7fe640 1 -- 172.21.15.7:0/1284585844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2e70047460 con 0x7f2e880b29d0 2024-09-15T23:37:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.439+0000 7f2e7f7fe640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2e6806c2b0 0x7f2e6806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.439+0000 7f2e7f7fe640 1 -- 172.21.15.7:0/1284585844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6590+0+0 (secure 0 0 0) 0x7f2e700ac300 con 0x7f2e880b29d0 2024-09-15T23:37:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.439+0000 7f2e86ffd640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2e6806c2b0 0x7f2e6806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.440+0000 7f2e86ffd640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2e6806c2b0 0x7f2e6806e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e78008ee0 tx=0x7f2e78008430 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.444+0000 7f2e7f7fe640 1 -- 172.21.15.7:0/1284585844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e70045360 con 0x7f2e880b29d0 2024-09-15T23:37:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.619+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e880b3a40 con 0x7f2e880b29d0 2024-09-15T23:37:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.620+0000 7f2e7f7fe640 1 -- 172.21.15.7:0/1284585844 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18601 (secure 0 0 0) 0x7f2e7007a790 con 0x7f2e880b29d0 2024-09-15T23:37:04.622 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:04.622 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:37:03.933844+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:36:20.781569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:36:21.898266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T23:36:25.611819+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1b","mappings":[{"from":0,"to":7}]},{"pgid":"4.e","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:37:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2e6806c2b0 msgr2=0x7f2e6806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2e6806c2b0 0x7f2e6806e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e78008ee0 tx=0x7f2e78008430 comp rx=0 tx=0).stop 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880b29d0 msgr2=0x7f2e8809d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880b29d0 0x7f2e8809d1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2e70000c00 tx=0x7f2e700046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 shutdown_connections 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e8815e800 0x7f2e8806bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880b29d0 0x7f2e8809d1f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2e6806c2b0 0x7f2e6806e770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 --2- 172.21.15.7:0/1284585844 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e880ae250 0x7f2e88151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.622+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 >> 172.21.15.7:0/1284585844 conn(0x7f2e88073490 msgr2=0x7f2e8806f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.623+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 shutdown_connections 2024-09-15T23:37:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.623+0000 7f2e8d766640 1 -- 172.21.15.7:0/1284585844 wait complete. 2024-09-15T23:37:04.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-15T23:37:04.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.737+0000 7f5d45877640 1 Processor -- start 2024-09-15T23:37:04.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.737+0000 7f5d45877640 1 -- start start 2024-09-15T23:37:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.737+0000 7f5d45877640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d40159e30 0x7f5d4006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.737+0000 7f5d45877640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 0x7f5d4015d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.737+0000 7f5d45877640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4006e480 con 0x7f5d400ad140 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d45877640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4006f940 con 0x7f5d40159e30 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d45877640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4006e760 con 0x7f5d4006f520 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d3effd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d40159e30 0x7f5d4006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d3effd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d40159e30 0x7f5d4006dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60518/0 (socket says 172.21.15.7:60518) 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d3effd640 1 -- 172.21.15.7:0/3004934907 learned_addr learned my addr 172.21.15.7:0/3004934907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d367fc640 1 --2- 172.21.15.7:0/3004934907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 0x7f5d4015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d3effd640 1 -- 172.21.15.7:0/3004934907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 msgr2=0x7f5d4015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d3effd640 1 --2- 172.21.15.7:0/3004934907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 0x7f5d4015d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d3effd640 1 -- 172.21.15.7:0/3004934907 >> v1:172.21.15.18:6789/0 conn(0x7f5d400ad140 legacy=0x7f5d400b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.738+0000 7f5d3effd640 1 -- 172.21.15.7:0/3004934907 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4006ea40 con 0x7f5d40159e30 2024-09-15T23:37:04.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d3effd640 1 --2- 172.21.15.7:0/3004934907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 0x7f5d4006dea0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009870 tx=0x7f5d3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb058ca4e696ea51 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:04.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d3e7fc640 1 -- 172.21.15.7:0/3004934907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30037070 con 0x7f5d40159e30 2024-09-15T23:37:04.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d3e7fc640 1 -- 172.21.15.7:0/3004934907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d30032440 con 0x7f5d40159e30 2024-09-15T23:37:04.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d3e7fc640 1 -- 172.21.15.7:0/3004934907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30032810 con 0x7f5d40159e30 2024-09-15T23:37:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d45877640 1 -- 172.21.15.7:0/3004934907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 msgr2=0x7f5d4006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d45877640 1 --2- 172.21.15.7:0/3004934907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 0x7f5d4006dea0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009870 tx=0x7f5d3002a270 comp rx=0 tx=0).stop 2024-09-15T23:37:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d45877640 1 -- 172.21.15.7:0/3004934907 shutdown_connections 2024-09-15T23:37:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d45877640 1 --2- 172.21.15.7:0/3004934907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 0x7f5d4015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d45877640 1 --2- 172.21.15.7:0/3004934907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 0x7f5d4006dea0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d45877640 1 -- 172.21.15.7:0/3004934907 >> 172.21.15.7:0/3004934907 conn(0x7f5d400b1e60 msgr2=0x7f5d400b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d45877640 1 -- 172.21.15.7:0/3004934907 shutdown_connections 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.739+0000 7f5d45877640 1 -- 172.21.15.7:0/3004934907 wait complete. 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.741+0000 7f5d45877640 1 Processor -- start 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.741+0000 7f5d45877640 1 -- start start 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.741+0000 7f5d45877640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 0x7f5d401f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.741+0000 7f5d45877640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d400ad140 0x7f5d40154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.741+0000 7f5d45877640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 0x7f5d40151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.741+0000 7f5d45877640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4015a210 con 0x7f5d40159e30 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.741+0000 7f5d45877640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4006f000 con 0x7f5d400ad140 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.741+0000 7f5d45877640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4015a390 con 0x7f5d4006f520 2024-09-15T23:37:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 0x7f5d40151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 0x7f5d40151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60534/0 (socket says 172.21.15.7:60534) 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3f7fe640 1 -- 172.21.15.7:0/2721245461 learned_addr learned my addr 172.21.15.7:0/2721245461 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d367fc640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d400ad140 0x7f5d40154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3effd640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 0x7f5d401f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3f7fe640 1 -- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 msgr2=0x7f5d401f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3f7fe640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 0x7f5d401f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3f7fe640 1 -- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d400ad140 msgr2=0x7f5d40154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3f7fe640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d400ad140 0x7f5d40154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3f7fe640 1 -- 172.21.15.7:0/2721245461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d40154df0 con 0x7f5d40159e30 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d367fc640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d400ad140 0x7f5d40154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3effd640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 0x7f5d401f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3f7fe640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 0x7f5d40151df0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c730 tx=0x7f5d3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3cff9640 1 -- 172.21.15.7:0/2721245461 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38016020 con 0x7f5d40159e30 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3cff9640 1 -- 172.21.15.7:0/2721245461 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d38006d50 con 0x7f5d40159e30 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d401550d0 con 0x7f5d40159e30 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d3cff9640 1 -- 172.21.15.7:0/2721245461 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38003bb0 con 0x7f5d40159e30 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.742+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d40155690 con 0x7f5d40159e30 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.743+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4006dea0 con 0x7f5d40159e30 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.744+0000 7f5d3cff9640 1 -- 172.21.15.7:0/2721245461 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5d380050d0 con 0x7f5d40159e30 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.744+0000 7f5d3cff9640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5d2006c2b0 0x7f5d2006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.745+0000 7f5d3cff9640 1 -- 172.21.15.7:0/2721245461 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6590+0+0 (secure 0 0 0) 0x7f5d3808b600 con 0x7f5d40159e30 2024-09-15T23:37:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.745+0000 7f5d3effd640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5d2006c2b0 0x7f5d2006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:04.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.745+0000 7f5d3effd640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5d2006c2b0 0x7f5d2006e770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d30034000 tx=0x7f5d3002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:04.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.749+0000 7f5d3cff9640 1 -- 172.21.15.7:0/2721245461 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5d38020020 con 0x7f5d40159e30 2024-09-15T23:37:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.967+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5d400b0150 con 0x7f5d40159e30 2024-09-15T23:37:04.969 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-15T23:37:04.967+0000 7f6dee804640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T23:37:04.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.969+0000 7f5d3cff9640 1 -- 172.21.15.7:0/2721245461 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f5d380079e0 con 0x7f5d40159e30 2024-09-15T23:37:04.970 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-15T23:37:04.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5d2006c2b0 msgr2=0x7f5d2006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5d2006c2b0 0x7f5d2006e770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d30034000 tx=0x7f5d3002e000 comp rx=0 tx=0).stop 2024-09-15T23:37:04.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 msgr2=0x7f5d40151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:04.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 0x7f5d40151df0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c730 tx=0x7f5d3800cbf0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 shutdown_connections 2024-09-15T23:37:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d40159e30 0x7f5d40151df0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d400ad140 0x7f5d40154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5d2006c2b0 0x7f5d2006e770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 --2- 172.21.15.7:0/2721245461 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d4006f520 0x7f5d401f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 >> 172.21.15.7:0/2721245461 conn(0x7f5d400b1e60 msgr2=0x7f5d4015b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 shutdown_connections 2024-09-15T23:37:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:04.971+0000 7f5d45877640 1 -- 172.21.15.7:0/2721245461 wait complete. 2024-09-15T23:37:04.978 INFO:tasks.ceph.mds.d.smithi018.stderr: -4209> 2024-09-15T23:36:28.540+0000 7f9d546c2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T23:37:04.986 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-15T23:37:05.011 INFO:tasks.ceph.mds.d.smithi018.stdout:starting mds.d at 2024-09-15T23:37:05.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.089+0000 7f7b8c3e4640 1 Processor -- start 2024-09-15T23:37:05.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.089+0000 7f7b8c3e4640 1 -- start start 2024-09-15T23:37:05.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8c3e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b84161f20 0x7f7b840d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:05.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8c3e4640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b840b5c40 0x7f7b840d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:05.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8c3e4640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b840d4f80 con 0x7f7b840b5570 2024-09-15T23:37:05.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8c3e4640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7b840d5540 con 0x7f7b840b5c40 2024-09-15T23:37:05.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8c3e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7b840d4d30 con 0x7f7b84161f20 2024-09-15T23:37:05.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8a981640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b840b5c40 0x7f7b840d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:05.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8a981640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b840b5c40 0x7f7b840d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60550/0 (socket says 172.21.15.7:60550) 2024-09-15T23:37:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8a981640 1 -- 172.21.15.7:0/3614546801 learned_addr learned my addr 172.21.15.7:0/3614546801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8a180640 1 --2- 172.21.15.7:0/3614546801 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b84161f20 0x7f7b840d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8a981640 1 -- 172.21.15.7:0/3614546801 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b84161f20 msgr2=0x7f7b840d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8a981640 1 --2- 172.21.15.7:0/3614546801 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b84161f20 0x7f7b840d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8a981640 1 -- 172.21.15.7:0/3614546801 >> v1:172.21.15.18:6789/0 conn(0x7f7b840b5570 legacy=0x7f7b840d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.090+0000 7f7b8a981640 1 -- 172.21.15.7:0/3614546801 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b840d5260 con 0x7f7b840b5c40 2024-09-15T23:37:05.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.091+0000 7f7b8a981640 1 --2- 172.21.15.7:0/3614546801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b840b5c40 0x7f7b840d1340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009e50 tx=0x7f7b7c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bceacbd8c24ae84 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:05.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.091+0000 7f7b8917e640 1 -- 172.21.15.7:0/3614546801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c036070 con 0x7f7b840b5c40 2024-09-15T23:37:05.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.091+0000 7f7b8917e640 1 -- 172.21.15.7:0/3614546801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7c031600 con 0x7f7b840b5c40 2024-09-15T23:37:05.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.091+0000 7f7b8917e640 1 -- 172.21.15.7:0/3614546801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0319d0 con 0x7f7b840b5c40 2024-09-15T23:37:05.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.092+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/3614546801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b840b5c40 msgr2=0x7f7b840d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:05.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.092+0000 7f7b8c3e4640 1 --2- 172.21.15.7:0/3614546801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b840b5c40 0x7f7b840d1340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009e50 tx=0x7f7b7c02a710 comp rx=0 tx=0).stop 2024-09-15T23:37:05.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.092+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/3614546801 shutdown_connections 2024-09-15T23:37:05.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.092+0000 7f7b8c3e4640 1 --2- 172.21.15.7:0/3614546801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b840b5c40 0x7f7b840d1340 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:05.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.092+0000 7f7b8c3e4640 1 --2- 172.21.15.7:0/3614546801 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b84161f20 0x7f7b840d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:05.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.092+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/3614546801 >> 172.21.15.7:0/3614546801 conn(0x7f7b8408bfd0 msgr2=0x7f7b840d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:05.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.092+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/3614546801 shutdown_connections 2024-09-15T23:37:05.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.092+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/3614546801 wait complete. 2024-09-15T23:37:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.093+0000 7f7b8c3e4640 1 Processor -- start 2024-09-15T23:37:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.093+0000 7f7b8c3e4640 1 -- start start 2024-09-15T23:37:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8c3e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b840b5570 0x7f7b840a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8c3e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b840b5c40 0x7f7b841f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8c3e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84161f20 0x7f7b840cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8c3e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b840b38f0 con 0x7f7b84161f20 2024-09-15T23:37:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8c3e4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7b840d0270 con 0x7f7b840b5570 2024-09-15T23:37:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8c3e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7b840b4c90 con 0x7f7b840b5c40 2024-09-15T23:37:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a981640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84161f20 0x7f7b840cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a981640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84161f20 0x7f7b840cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60566/0 (socket says 172.21.15.7:60566) 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a981640 1 -- 172.21.15.7:0/1342176673 learned_addr learned my addr 172.21.15.7:0/1342176673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a180640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b840b5570 0x7f7b840a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8997f640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b840b5c40 0x7f7b841f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a981640 1 -- 172.21.15.7:0/1342176673 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b840b5c40 msgr2=0x7f7b841f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a981640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b840b5c40 0x7f7b841f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a981640 1 -- 172.21.15.7:0/1342176673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b840b5570 msgr2=0x7f7b840a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a981640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b840b5570 0x7f7b840a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a981640 1 -- 172.21.15.7:0/1342176673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b841f4fa0 con 0x7f7b84161f20 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8a981640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84161f20 0x7f7b840cdd00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009e50 tx=0x7f7b7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b7affd640 1 -- 172.21.15.7:0/1342176673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c036050 con 0x7f7b84161f20 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b7affd640 1 -- 172.21.15.7:0/1342176673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7c042070 con 0x7f7b84161f20 2024-09-15T23:37:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b841f5280 con 0x7f7b84161f20 2024-09-15T23:37:05.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b7affd640 1 -- 172.21.15.7:0/1342176673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c009510 con 0x7f7b84161f20 2024-09-15T23:37:05.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.094+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b840b9180 con 0x7f7b84161f20 2024-09-15T23:37:05.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.095+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b840d0e00 con 0x7f7b84161f20 2024-09-15T23:37:05.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.096+0000 7f7b7affd640 1 -- 172.21.15.7:0/1342176673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7b7c007bd0 con 0x7f7b84161f20 2024-09-15T23:37:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.096+0000 7f7b7affd640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7b6806c2e0 0x7f7b6806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.097+0000 7f7b7affd640 1 -- 172.21.15.7:0/1342176673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6676+0+0 (secure 0 0 0) 0x7f7b7c0ac530 con 0x7f7b84161f20 2024-09-15T23:37:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.097+0000 7f7b8a180640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7b6806c2e0 0x7f7b6806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.097+0000 7f7b8a180640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7b6806c2e0 0x7f7b6806e7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.101+0000 7f7b7affd640 1 -- 172.21.15.7:0/1342176673 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7b7c07a290 con 0x7f7b84161f20 2024-09-15T23:37:05.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.317+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7b840b8a70 con 0x7f7b84161f20 2024-09-15T23:37:05.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.972+0000 7f7b7affd640 1 -- 172.21.15.7:0/1342176673 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f7b7c07e590 con 0x7f7b84161f20 2024-09-15T23:37:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7b6806c2e0 msgr2=0x7f7b6806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7b6806c2e0 0x7f7b6806e7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80008040 comp rx=0 tx=0).stop 2024-09-15T23:37:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84161f20 msgr2=0x7f7b840cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84161f20 0x7f7b840cdd00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009e50 tx=0x7f7b7c004a40 comp rx=0 tx=0).stop 2024-09-15T23:37:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 shutdown_connections 2024-09-15T23:37:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84161f20 0x7f7b840cdd00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f7b6806c2e0 0x7f7b6806e7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b840b5c40 0x7f7b841f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 --2- 172.21.15.7:0/1342176673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b840b5570 0x7f7b840a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 >> 172.21.15.7:0/1342176673 conn(0x7f7b8408bfd0 msgr2=0x7f7b8415fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 shutdown_connections 2024-09-15T23:37:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:05.975+0000 7f7b8c3e4640 1 -- 172.21.15.7:0/1342176673 wait complete. 2024-09-15T23:37:05.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-15T23:37:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.099+0000 7f3f3dd66640 1 Processor -- start 2024-09-15T23:37:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.099+0000 7f3f3dd66640 1 -- start start 2024-09-15T23:37:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.100+0000 7f3f3dd66640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3f38161cb0 0x7f3f380b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.100+0000 7f3f3dd66640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3f38162a30 0x7f3f380b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.100+0000 7f3f3dd66640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f38162090 con 0x7f3f380b1870 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.100+0000 7f3f3dd66640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3f380b4b40 con 0x7f3f38161cb0 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.100+0000 7f3f3dd66640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3f380b3f30 con 0x7f3f38162a30 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.100+0000 7f3f377fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3f38161cb0 0x7f3f380b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.100+0000 7f3f377fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3f38161cb0 0x7f3f380b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60576/0 (socket says 172.21.15.7:60576) 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.100+0000 7f3f377fe640 1 -- 172.21.15.7:0/2432752373 learned_addr learned my addr 172.21.15.7:0/2432752373 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.100+0000 7f3f37fff640 1 --2- 172.21.15.7:0/2432752373 >> v2:172.21.15.18:3300/0 conn(0x7f3f38162a30 0x7f3f380b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.101+0000 7f3f377fe640 1 -- 172.21.15.7:0/2432752373 >> v2:172.21.15.18:3300/0 conn(0x7f3f38162a30 msgr2=0x7f3f380b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.101+0000 7f3f377fe640 1 --2- 172.21.15.7:0/2432752373 >> v2:172.21.15.18:3300/0 conn(0x7f3f38162a30 0x7f3f380b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.101+0000 7f3f377fe640 1 -- 172.21.15.7:0/2432752373 >> v1:172.21.15.18:6789/0 conn(0x7f3f380b1870 legacy=0x7f3f38162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.101+0000 7f3f377fe640 1 -- 172.21.15.7:0/2432752373 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f380b4210 con 0x7f3f38161cb0 2024-09-15T23:37:06.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.101+0000 7f3f377fe640 1 --2- 172.21.15.7:0/2432752373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38161cb0 0x7f3f380b1330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3f3808def0 tx=0x7f3f2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b255d40d7868b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f367fc640 1 -- 172.21.15.7:0/2432752373 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28037070 con 0x7f3f38161cb0 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f367fc640 1 -- 172.21.15.7:0/2432752373 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f28032440 con 0x7f3f38161cb0 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f367fc640 1 -- 172.21.15.7:0/2432752373 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28032810 con 0x7f3f38161cb0 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f3dd66640 1 -- 172.21.15.7:0/2432752373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38161cb0 msgr2=0x7f3f380b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f3dd66640 1 --2- 172.21.15.7:0/2432752373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38161cb0 0x7f3f380b1330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3f3808def0 tx=0x7f3f2802a270 comp rx=0 tx=0).stop 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f3dd66640 1 -- 172.21.15.7:0/2432752373 shutdown_connections 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f3dd66640 1 --2- 172.21.15.7:0/2432752373 >> v2:172.21.15.18:3300/0 conn(0x7f3f38162a30 0x7f3f380b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f3dd66640 1 --2- 172.21.15.7:0/2432752373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38161cb0 0x7f3f380b1330 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f3dd66640 1 -- 172.21.15.7:0/2432752373 >> 172.21.15.7:0/2432752373 conn(0x7f3f3808bfd0 msgr2=0x7f3f380d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f3dd66640 1 -- 172.21.15.7:0/2432752373 shutdown_connections 2024-09-15T23:37:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.102+0000 7f3f3dd66640 1 -- 172.21.15.7:0/2432752373 wait complete. 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.103+0000 7f3f3dd66640 1 Processor -- start 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f3dd66640 1 -- start start 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f3dd66640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3f380b1870 0x7f3f380c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f3dd66640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f38161cb0 0x7f3f380ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f3dd66640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38162a30 0x7f3f380d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f3dd66640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f380b3bc0 con 0x7f3f38162a30 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f3dd66640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3f380b47d0 con 0x7f3f38161cb0 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f3dd66640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f3f38163000 con 0x7f3f380b1870 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38162a30 0x7f3f380d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38162a30 0x7f3f380d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60590/0 (socket says 172.21.15.7:60590) 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.104+0000 7f3f37fff640 1 -- 172.21.15.7:0/236622661 learned_addr learned my addr 172.21.15.7:0/236622661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f36ffd640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f38161cb0 0x7f3f380ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f377fe640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3f380b1870 0x7f3f380c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f37fff640 1 -- 172.21.15.7:0/236622661 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3f380b1870 msgr2=0x7f3f380c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f37fff640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3f380b1870 0x7f3f380c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f37fff640 1 -- 172.21.15.7:0/236622661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f38161cb0 msgr2=0x7f3f380ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f37fff640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f38161cb0 0x7f3f380ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f37fff640 1 -- 172.21.15.7:0/236622661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f380c87a0 con 0x7f3f38162a30 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f36ffd640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f38161cb0 0x7f3f380ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f37fff640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38162a30 0x7f3f380d13e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f30007d80 tx=0x7f3f3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f1ffff640 1 -- 172.21.15.7:0/236622661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30014420 con 0x7f3f38162a30 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f1ffff640 1 -- 172.21.15.7:0/236622661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f30019070 con 0x7f3f38162a30 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f380c8a80 con 0x7f3f38162a30 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f1ffff640 1 -- 172.21.15.7:0/236622661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30014800 con 0x7f3f38162a30 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.105+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f380c9040 con 0x7f3f38162a30 2024-09-15T23:37:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.106+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f380b1330 con 0x7f3f38162a30 2024-09-15T23:37:06.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.107+0000 7f3f1ffff640 1 -- 172.21.15.7:0/236622661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3f300149a0 con 0x7f3f38162a30 2024-09-15T23:37:06.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.107+0000 7f3f1ffff640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f3f2406c2b0 0x7f3f2406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:06.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.107+0000 7f3f1ffff640 1 -- 172.21.15.7:0/236622661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6676+0+0 (secure 0 0 0) 0x7f3f3008bfd0 con 0x7f3f38162a30 2024-09-15T23:37:06.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.107+0000 7f3f377fe640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f3f2406c2b0 0x7f3f2406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:06.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.108+0000 7f3f377fe640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f3f2406c2b0 0x7f3f2406e770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3f28034000 tx=0x7f3f2802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:06.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.112+0000 7f3f1ffff640 1 -- 172.21.15.7:0/236622661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3f3005a400 con 0x7f3f38162a30 2024-09-15T23:37:06.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.287+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3f38160f40 con 0x7f3f38162a30 2024-09-15T23:37:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.974+0000 7f3f1ffff640 1 -- 172.21.15.7:0/236622661 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f3f30059d00 con 0x7f3f38162a30 2024-09-15T23:37:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-15T23:37:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.977+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f3f2406c2b0 msgr2=0x7f3f2406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.977+0000 7f3f3dd66640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f3f2406c2b0 0x7f3f2406e770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3f28034000 tx=0x7f3f2802e000 comp rx=0 tx=0).stop 2024-09-15T23:37:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.977+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38162a30 msgr2=0x7f3f380d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.977+0000 7f3f3dd66640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38162a30 0x7f3f380d13e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f30007d80 tx=0x7f3f3000e4b0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.978+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 shutdown_connections 2024-09-15T23:37:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.978+0000 7f3f3dd66640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f38162a30 0x7f3f380d13e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.978+0000 7f3f3dd66640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f38161cb0 0x7f3f380ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.978+0000 7f3f3dd66640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f3f2406c2b0 0x7f3f2406e770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.978+0000 7f3f3dd66640 1 --2- 172.21.15.7:0/236622661 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3f380b1870 0x7f3f380c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.978+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 >> 172.21.15.7:0/236622661 conn(0x7f3f3808bfd0 msgr2=0x7f3f380b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.979+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 shutdown_connections 2024-09-15T23:37:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:06.980+0000 7f3f3dd66640 1 -- 172.21.15.7:0/236622661 wait complete. 2024-09-15T23:37:06.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-15T23:37:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd7313e4640 1 Processor -- start 2024-09-15T23:37:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd7313e4640 1 -- start start 2024-09-15T23:37:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd7313e4640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd72c15fa70 0x7fd72c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd7313e4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd72c15fe50 0x7fd72c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd7313e4640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72c0701d0 con 0x7fd72c0d3060 2024-09-15T23:37:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd7313e4640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd72c0716c0 con 0x7fd72c15fa70 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd7313e4640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fd72c09ba40 con 0x7fd72c15fe50 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd72affd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd72c15fa70 0x7fd72c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd72affd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd72c15fa70 0x7fd72c0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53388/0 (socket says 172.21.15.7:53388) 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd72affd640 1 -- 172.21.15.7:0/3010758278 learned_addr learned my addr 172.21.15.7:0/3010758278 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd72a7fc640 1 --2- 172.21.15.7:0/3010758278 >> v2:172.21.15.18:3300/0 conn(0x7fd72c15fe50 0x7fd72c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd72affd640 1 -- 172.21.15.7:0/3010758278 >> v2:172.21.15.18:3300/0 conn(0x7fd72c15fe50 msgr2=0x7fd72c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd72affd640 1 --2- 172.21.15.7:0/3010758278 >> v2:172.21.15.18:3300/0 conn(0x7fd72c15fe50 0x7fd72c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd72affd640 1 -- 172.21.15.7:0/3010758278 >> v1:172.21.15.18:6789/0 conn(0x7fd72c0d3060 legacy=0x7fd72c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd72affd640 1 -- 172.21.15.7:0/3010758278 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c09bd20 con 0x7fd72c15fa70 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.109+0000 7fd72a7fc640 1 --2- 172.21.15.7:0/3010758278 >> v2:172.21.15.18:3300/0 conn(0x7fd72c15fe50 0x7fd72c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.110+0000 7fd72affd640 1 --2- 172.21.15.7:0/3010758278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fa70 0x7fd72c0cce50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0740c0 tx=0x7fd71402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b0f2499d24fef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.110+0000 7fd729ffb640 1 -- 172.21.15.7:0/3010758278 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714037070 con 0x7fd72c15fa70 2024-09-15T23:37:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.110+0000 7fd729ffb640 1 -- 172.21.15.7:0/3010758278 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd714032440 con 0x7fd72c15fa70 2024-09-15T23:37:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.110+0000 7fd729ffb640 1 -- 172.21.15.7:0/3010758278 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714032810 con 0x7fd72c15fa70 2024-09-15T23:37:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.111+0000 7fd7313e4640 1 -- 172.21.15.7:0/3010758278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fa70 msgr2=0x7fd72c0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.111+0000 7fd7313e4640 1 --2- 172.21.15.7:0/3010758278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fa70 0x7fd72c0cce50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0740c0 tx=0x7fd71402a230 comp rx=0 tx=0).stop 2024-09-15T23:37:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.111+0000 7fd7313e4640 1 -- 172.21.15.7:0/3010758278 shutdown_connections 2024-09-15T23:37:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.111+0000 7fd7313e4640 1 --2- 172.21.15.7:0/3010758278 >> v2:172.21.15.18:3300/0 conn(0x7fd72c15fe50 0x7fd72c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.111+0000 7fd7313e4640 1 --2- 172.21.15.7:0/3010758278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fa70 0x7fd72c0cce50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.111+0000 7fd7313e4640 1 -- 172.21.15.7:0/3010758278 >> 172.21.15.7:0/3010758278 conn(0x7fd72c0d2420 msgr2=0x7fd72c0d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.111+0000 7fd7313e4640 1 -- 172.21.15.7:0/3010758278 shutdown_connections 2024-09-15T23:37:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.111+0000 7fd7313e4640 1 -- 172.21.15.7:0/3010758278 wait complete. 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.112+0000 7fd7313e4640 1 Processor -- start 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.112+0000 7fd7313e4640 1 -- start start 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd7313e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd72c0d3060 0x7fd72c1f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd7313e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd72c15fa70 0x7fd72c15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd7313e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fe50 0x7fd72c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd7313e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c09b2d0 con 0x7fd72c15fe50 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd7313e4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c09b150 con 0x7fd72c0d3060 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd7313e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c071ef0 con 0x7fd72c15fa70 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fe50 0x7fd72c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fe50 0x7fd72c15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53402/0 (socket says 172.21.15.7:53402) 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72b7fe640 1 -- 172.21.15.7:0/929296617 learned_addr learned my addr 172.21.15.7:0/929296617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:07.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72affd640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd72c0d3060 0x7fd72c1f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72a7fc640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd72c15fa70 0x7fd72c15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72b7fe640 1 -- 172.21.15.7:0/929296617 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd72c15fa70 msgr2=0x7fd72c15acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72b7fe640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd72c15fa70 0x7fd72c15acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72b7fe640 1 -- 172.21.15.7:0/929296617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd72c0d3060 msgr2=0x7fd72c1f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72b7fe640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd72c0d3060 0x7fd72c1f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72b7fe640 1 -- 172.21.15.7:0/929296617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c0cc5b0 con 0x7fd72c15fe50 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72a7fc640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd72c15fa70 0x7fd72c15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.113+0000 7fd72b7fe640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fe50 0x7fd72c15b1f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c780 tx=0x7fd72400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.114+0000 7fd7237fe640 1 -- 172.21.15.7:0/929296617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724004b60 con 0x7fd72c15fe50 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.114+0000 7fd7237fe640 1 -- 172.21.15.7:0/929296617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd724004d00 con 0x7fd72c15fe50 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.114+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c0cb2e0 con 0x7fd72c15fe50 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.114+0000 7fd7237fe640 1 -- 172.21.15.7:0/929296617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72400aa90 con 0x7fd72c15fe50 2024-09-15T23:37:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.114+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c0cc260 con 0x7fd72c15fe50 2024-09-15T23:37:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.115+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c0cb9a0 con 0x7fd72c15fe50 2024-09-15T23:37:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.115+0000 7fd7237fe640 1 -- 172.21.15.7:0/929296617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd72400ac30 con 0x7fd72c15fe50 2024-09-15T23:37:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.116+0000 7fd7237fe640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd70c06c2b0 0x7fd70c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.116+0000 7fd7237fe640 1 -- 172.21.15.7:0/929296617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6238+0+0 (secure 0 0 0) 0x7fd72408b870 con 0x7fd72c15fe50 2024-09-15T23:37:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.116+0000 7fd72affd640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd70c06c2b0 0x7fd70c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.116+0000 7fd72affd640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd70c06c2b0 0x7fd70c06e770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd714034000 tx=0x7fd71402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.121+0000 7fd7237fe640 1 -- 172.21.15.7:0/929296617 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd72c0cb9a0 con 0x7fd72c15fe50 2024-09-15T23:37:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.295+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd72c09aa40 con 0x7fd72c15fe50 2024-09-15T23:37:07.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.979+0000 7fd7237fe640 1 -- 172.21.15.7:0/929296617 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7fd724020070 con 0x7fd72c15fe50 2024-09-15T23:37:07.980 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-15T23:37:07.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.983+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd70c06c2b0 msgr2=0x7fd70c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:07.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.983+0000 7fd7313e4640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd70c06c2b0 0x7fd70c06e770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd714034000 tx=0x7fd71402e000 comp rx=0 tx=0).stop 2024-09-15T23:37:07.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.983+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fe50 msgr2=0x7fd72c15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:07.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.983+0000 7fd7313e4640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fe50 0x7fd72c15b1f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c780 tx=0x7fd72400cc40 comp rx=0 tx=0).stop 2024-09-15T23:37:07.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.984+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 shutdown_connections 2024-09-15T23:37:07.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.984+0000 7fd7313e4640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd72c15fe50 0x7fd72c15b1f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:07.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.984+0000 7fd7313e4640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fd70c06c2b0 0x7fd70c06e770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:07.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.984+0000 7fd7313e4640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd72c15fa70 0x7fd72c15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:07.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.984+0000 7fd7313e4640 1 --2- 172.21.15.7:0/929296617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd72c0d3060 0x7fd72c1f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:07.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.984+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 >> 172.21.15.7:0/929296617 conn(0x7fd72c0d2420 msgr2=0x7fd72c0d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:07.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.985+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 shutdown_connections 2024-09-15T23:37:07.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:07.985+0000 7fd7313e4640 1 -- 172.21.15.7:0/929296617 wait complete. 2024-09-15T23:37:08.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-15T23:37:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.114+0000 7f994c70a640 1 Processor -- start 2024-09-15T23:37:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.114+0000 7f994c70a640 1 -- start start 2024-09-15T23:37:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f994c70a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9944070ea0 0x7f99440d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f994c70a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 0x7f99440b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f994c70a640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994409bae0 con 0x7f994415e850 2024-09-15T23:37:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f994c70a640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f99440d05b0 con 0x7f9944070ea0 2024-09-15T23:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f994c70a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f99440d1110 con 0x7f994409b450 2024-09-15T23:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f9949ca5640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9944070ea0 0x7f99440d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f9949ca5640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9944070ea0 0x7f99440d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53418/0 (socket says 172.21.15.7:53418) 2024-09-15T23:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f9949ca5640 1 -- 172.21.15.7:0/3562380558 learned_addr learned my addr 172.21.15.7:0/3562380558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f994aca7640 1 --2- 172.21.15.7:0/3562380558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 0x7f99440b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f9949ca5640 1 -- 172.21.15.7:0/3562380558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 msgr2=0x7f99440b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f9949ca5640 1 --2- 172.21.15.7:0/3562380558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 0x7f99440b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f9949ca5640 1 -- 172.21.15.7:0/3562380558 >> v1:172.21.15.18:6789/0 conn(0x7f994415e850 legacy=0x7f9944070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.115+0000 7f9949ca5640 1 -- 172.21.15.7:0/3562380558 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99440d07e0 con 0x7f9944070ea0 2024-09-15T23:37:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.116+0000 7f9949ca5640 1 --2- 172.21.15.7:0/3562380558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9944070ea0 0x7f99440d15a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9934009870 tx=0x7f993402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3233c1e10104be server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.116+0000 7f99494a4640 1 -- 172.21.15.7:0/3562380558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934037070 con 0x7f9944070ea0 2024-09-15T23:37:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.116+0000 7f99494a4640 1 -- 172.21.15.7:0/3562380558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934032440 con 0x7f9944070ea0 2024-09-15T23:37:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.116+0000 7f99494a4640 1 -- 172.21.15.7:0/3562380558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934032810 con 0x7f9944070ea0 2024-09-15T23:37:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.117+0000 7f994c70a640 1 -- 172.21.15.7:0/3562380558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9944070ea0 msgr2=0x7f99440d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.117+0000 7f994c70a640 1 --2- 172.21.15.7:0/3562380558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9944070ea0 0x7f99440d15a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9934009870 tx=0x7f993402a270 comp rx=0 tx=0).stop 2024-09-15T23:37:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.117+0000 7f994c70a640 1 -- 172.21.15.7:0/3562380558 shutdown_connections 2024-09-15T23:37:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.117+0000 7f994c70a640 1 --2- 172.21.15.7:0/3562380558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 0x7f99440b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.117+0000 7f994c70a640 1 --2- 172.21.15.7:0/3562380558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9944070ea0 0x7f99440d15a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.117+0000 7f994c70a640 1 -- 172.21.15.7:0/3562380558 >> 172.21.15.7:0/3562380558 conn(0x7f99440d2420 msgr2=0x7f99440d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.117+0000 7f994c70a640 1 -- 172.21.15.7:0/3562380558 shutdown_connections 2024-09-15T23:37:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.117+0000 7f994c70a640 1 -- 172.21.15.7:0/3562380558 wait complete. 2024-09-15T23:37:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.118+0000 7f994c70a640 1 Processor -- start 2024-09-15T23:37:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.118+0000 7f994c70a640 1 -- start start 2024-09-15T23:37:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994c70a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9944070ea0 0x7f99441f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994c70a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 0x7f9944157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994c70a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994415e850 0x7f99441f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994c70a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f994415ec30 con 0x7f994415e850 2024-09-15T23:37:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994c70a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99440d0da0 con 0x7f9944070ea0 2024-09-15T23:37:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994c70a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f994406cca0 con 0x7f994409b450 2024-09-15T23:37:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994aca7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994415e850 0x7f99441f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994aca7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994415e850 0x7f99441f6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53432/0 (socket says 172.21.15.7:53432) 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994aca7640 1 -- 172.21.15.7:0/2189735392 learned_addr learned my addr 172.21.15.7:0/2189735392 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f9949ca5640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 0x7f9944157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994a4a6640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9944070ea0 0x7f99441f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994aca7640 1 -- 172.21.15.7:0/2189735392 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 msgr2=0x7f9944157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994aca7640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 0x7f9944157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994aca7640 1 -- 172.21.15.7:0/2189735392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9944070ea0 msgr2=0x7f99441f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994aca7640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9944070ea0 0x7f99441f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.119+0000 7f994aca7640 1 -- 172.21.15.7:0/2189735392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9944159560 con 0x7f994415e850 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.120+0000 7f994aca7640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994415e850 0x7f99441f6190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f993c00efc0 tx=0x7f993c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.120+0000 7f993affd640 1 -- 172.21.15.7:0/2189735392 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c018020 con 0x7f994415e850 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.120+0000 7f993affd640 1 -- 172.21.15.7:0/2189735392 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c014480 con 0x7f994415e850 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.120+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99441f5740 con 0x7f994415e850 2024-09-15T23:37:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.120+0000 7f993affd640 1 -- 172.21.15.7:0/2189735392 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c014870 con 0x7f994415e850 2024-09-15T23:37:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.120+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99441f5d00 con 0x7f994415e850 2024-09-15T23:37:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.121+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99440d15a0 con 0x7f994415e850 2024-09-15T23:37:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.121+0000 7f993affd640 1 -- 172.21.15.7:0/2189735392 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f993c005d10 con 0x7f994415e850 2024-09-15T23:37:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.122+0000 7f993affd640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f992806c2b0 0x7f992806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:08.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.122+0000 7f993affd640 1 -- 172.21.15.7:0/2189735392 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5795+0+0 (secure 0 0 0) 0x7f993c08be70 con 0x7f994415e850 2024-09-15T23:37:08.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.122+0000 7f994a4a6640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f992806c2b0 0x7f992806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:08.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.122+0000 7f994a4a6640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f992806c2b0 0x7f992806e770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99441f8520 tx=0x7f9940009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:08.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.128+0000 7f993affd640 1 -- 172.21.15.7:0/2189735392 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f993c022070 con 0x7f994415e850 2024-09-15T23:37:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.303+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9944163180 con 0x7f994415e850 2024-09-15T23:37:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.984+0000 7f993affd640 1 -- 172.21.15.7:0/2189735392 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f993c010040 con 0x7f994415e850 2024-09-15T23:37:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' removed 2024-09-15T23:37:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.987+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f992806c2b0 msgr2=0x7f992806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.987+0000 7f994c70a640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f992806c2b0 0x7f992806e770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99441f8520 tx=0x7f9940009040 comp rx=0 tx=0).stop 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.987+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994415e850 msgr2=0x7f99441f6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.987+0000 7f994c70a640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994415e850 0x7f99441f6190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f993c00efc0 tx=0x7f993c00c490 comp rx=0 tx=0).stop 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.988+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 shutdown_connections 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.988+0000 7f994c70a640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994415e850 0x7f99441f6190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.988+0000 7f994c70a640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f992806c2b0 0x7f992806e770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.988+0000 7f994c70a640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f994409b450 0x7f9944157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.988+0000 7f994c70a640 1 --2- 172.21.15.7:0/2189735392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9944070ea0 0x7f99441f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.988+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 >> 172.21.15.7:0/2189735392 conn(0x7f99440d2420 msgr2=0x7f99440b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.988+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 shutdown_connections 2024-09-15T23:37:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:08.988+0000 7f994c70a640 1 -- 172.21.15.7:0/2189735392 wait complete. 2024-09-15T23:37:09.003 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-15T23:37:09.003 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-15T23:37:09.004 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-15T23:37:09.004 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:37:09.004 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-15T23:37:09.004 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-15T23:37:09.004 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-15T23:37:09.004 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:37:09.004 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-15T23:37:09.005 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-15T23:37:09.005 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-15T23:37:09.005 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:37:09.005 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-15T23:37:09.005 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-15T23:37:09.005 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-15T23:37:09.005 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:37:09.014 INFO:tasks.ceph.mds.c:Stopped 2024-09-15T23:37:09.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-15T23:37:09.015 INFO:tasks.ceph.mds.a:Stopped 2024-09-15T23:37:09.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-15T23:37:09.016 INFO:tasks.ceph.mds.b:Stopped 2024-09-15T23:37:09.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-15T23:37:09.017 INFO:tasks.ceph.mds.d:Stopped 2024-09-15T23:37:09.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-15T23:37:09.057 INFO:tasks.ceph.mds.c:Started 2024-09-15T23:37:09.058 DEBUG:teuthology.parallel:result is None 2024-09-15T23:37:09.059 INFO:tasks.ceph.mds.a:Started 2024-09-15T23:37:09.059 DEBUG:teuthology.parallel:result is None 2024-09-15T23:37:09.060 INFO:tasks.ceph.mds.b:Started 2024-09-15T23:37:09.060 DEBUG:teuthology.parallel:result is None 2024-09-15T23:37:09.061 INFO:tasks.ceph.mds.d:Started 2024-09-15T23:37:09.061 DEBUG:teuthology.parallel:result is None 2024-09-15T23:37:09.061 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-15T23:37:09.140 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-15T23:37:09.140 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-15T23:37:09.147 INFO:tasks.ceph.mds.d.smithi018.stdout:starting mds.d at 2024-09-15T23:37:09.148 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2024-09-15T23:37:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.172+0000 7f304ff11640 1 Processor -- start 2024-09-15T23:37:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.172+0000 7f304ff11640 1 -- start start 2024-09-15T23:37:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304ff11640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f304815e6f0 0x7f30480d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304ff11640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f30480d4f40 0x7f30480d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304ff11640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30480d2ef0 con 0x7f30480d1cb0 2024-09-15T23:37:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304ff11640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f30480d31d0 con 0x7f304815e6f0 2024-09-15T23:37:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304ff11640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f30480d34b0 con 0x7f30480d4f40 2024-09-15T23:37:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304dcad640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f304815e6f0 0x7f30480d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304dcad640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f304815e6f0 0x7f30480d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53492/0 (socket says 172.21.15.7:53492) 2024-09-15T23:37:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304dcad640 1 -- 172.21.15.7:0/1979787244 learned_addr learned my addr 172.21.15.7:0/1979787244 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304d4ac640 1 --2- 172.21.15.7:0/1979787244 >> v2:172.21.15.18:3300/0 conn(0x7f30480d4f40 0x7f30480d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304dcad640 1 -- 172.21.15.7:0/1979787244 >> v2:172.21.15.18:3300/0 conn(0x7f30480d4f40 msgr2=0x7f30480d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304dcad640 1 --2- 172.21.15.7:0/1979787244 >> v2:172.21.15.18:3300/0 conn(0x7f30480d4f40 0x7f30480d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304dcad640 1 -- 172.21.15.7:0/1979787244 >> v1:172.21.15.18:6789/0 conn(0x7f30480d1cb0 legacy=0x7f30480cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.173+0000 7f304dcad640 1 -- 172.21.15.7:0/1979787244 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304806ce60 con 0x7f304815e6f0 2024-09-15T23:37:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.174+0000 7f304dcad640 1 --2- 172.21.15.7:0/1979787244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 0x7f30480d0e60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30480d5700 tx=0x7f304402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515d5334c68f8661 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.174+0000 7f304ccab640 1 -- 172.21.15.7:0/1979787244 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044037070 con 0x7f304815e6f0 2024-09-15T23:37:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.174+0000 7f304ccab640 1 -- 172.21.15.7:0/1979787244 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3044032440 con 0x7f304815e6f0 2024-09-15T23:37:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.174+0000 7f304ccab640 1 -- 172.21.15.7:0/1979787244 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044032810 con 0x7f304815e6f0 2024-09-15T23:37:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.175+0000 7f304ff11640 1 -- 172.21.15.7:0/1979787244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 msgr2=0x7f30480d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.175+0000 7f304ff11640 1 --2- 172.21.15.7:0/1979787244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 0x7f30480d0e60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30480d5700 tx=0x7f304402a230 comp rx=0 tx=0).stop 2024-09-15T23:37:09.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.175+0000 7f304ff11640 1 -- 172.21.15.7:0/1979787244 shutdown_connections 2024-09-15T23:37:09.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.175+0000 7f304ff11640 1 --2- 172.21.15.7:0/1979787244 >> v2:172.21.15.18:3300/0 conn(0x7f30480d4f40 0x7f30480d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:09.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.175+0000 7f304ff11640 1 --2- 172.21.15.7:0/1979787244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 0x7f30480d0e60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:09.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.175+0000 7f304ff11640 1 -- 172.21.15.7:0/1979787244 >> 172.21.15.7:0/1979787244 conn(0x7f304808bfd0 msgr2=0x7f30480d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:09.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.175+0000 7f304ff11640 1 -- 172.21.15.7:0/1979787244 shutdown_connections 2024-09-15T23:37:09.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.175+0000 7f304ff11640 1 -- 172.21.15.7:0/1979787244 wait complete. 2024-09-15T23:37:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.176+0000 7f304ff11640 1 Processor -- start 2024-09-15T23:37:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.176+0000 7f304ff11640 1 -- start start 2024-09-15T23:37:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304ff11640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480d1cb0 0x7f30480cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304ff11640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30480d4f40 0x7f30480a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304ff11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 0x7f30480b9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304ff11640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f304815ead0 con 0x7f304815e6f0 2024-09-15T23:37:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304ff11640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f30480d2be0 con 0x7f30480d1cb0 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304ff11640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f304815ec50 con 0x7f30480d4f40 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304e4ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 0x7f30480b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304e4ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 0x7f30480b9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53508/0 (socket says 172.21.15.7:53508) 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304e4ae640 1 -- 172.21.15.7:0/1866219608 learned_addr learned my addr 172.21.15.7:0/1866219608 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304d4ac640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30480d4f40 0x7f30480a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304dcad640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480d1cb0 0x7f30480cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304e4ae640 1 -- 172.21.15.7:0/1866219608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30480d4f40 msgr2=0x7f30480a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304e4ae640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30480d4f40 0x7f30480a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304e4ae640 1 -- 172.21.15.7:0/1866219608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480d1cb0 msgr2=0x7f30480cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304e4ae640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480d1cb0 0x7f30480cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304e4ae640 1 -- 172.21.15.7:0/1866219608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30480a0630 con 0x7f304815e6f0 2024-09-15T23:37:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304e4ae640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 0x7f30480b9640 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f304000c780 tx=0x7f304000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:09.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f303e7fc640 1 -- 172.21.15.7:0/1866219608 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040004b60 con 0x7f304815e6f0 2024-09-15T23:37:09.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f303e7fc640 1 -- 172.21.15.7:0/1866219608 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3040004d00 con 0x7f304815e6f0 2024-09-15T23:37:09.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f303e7fc640 1 -- 172.21.15.7:0/1866219608 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304000aa90 con 0x7f304815e6f0 2024-09-15T23:37:09.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.177+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30480a0910 con 0x7f304815e6f0 2024-09-15T23:37:09.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.178+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30480a0ed0 con 0x7f304815e6f0 2024-09-15T23:37:09.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.178+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30480d0e60 con 0x7f304815e6f0 2024-09-15T23:37:09.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.179+0000 7f303e7fc640 1 -- 172.21.15.7:0/1866219608 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f304000ac30 con 0x7f304815e6f0 2024-09-15T23:37:09.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.180+0000 7f303e7fc640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f302c06c2b0 0x7f302c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:09.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.180+0000 7f304dcad640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f302c06c2b0 0x7f302c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:09.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.180+0000 7f303e7fc640 1 -- 172.21.15.7:0/1866219608 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7f304008b470 con 0x7f304815e6f0 2024-09-15T23:37:09.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.180+0000 7f304dcad640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f302c06c2b0 0x7f302c06e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3044034000 tx=0x7f304402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:09.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.184+0000 7f303e7fc640 1 -- 172.21.15.7:0/1866219608 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3040020070 con 0x7f304815e6f0 2024-09-15T23:37:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.373+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f30480b2710 con 0x7f304815e6f0 2024-09-15T23:37:09.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.995+0000 7f303e7fc640 1 -- 172.21.15.7:0/1866219608 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7f30400596f0 con 0x7f304815e6f0 2024-09-15T23:37:09.996 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-15T23:37:09.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.998+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f302c06c2b0 msgr2=0x7f302c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.998+0000 7f304ff11640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f302c06c2b0 0x7f302c06e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3044034000 tx=0x7f304402e000 comp rx=0 tx=0).stop 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.998+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 msgr2=0x7f30480b9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.998+0000 7f304ff11640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 0x7f30480b9640 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f304000c780 tx=0x7f304000cc40 comp rx=0 tx=0).stop 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.999+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 shutdown_connections 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.999+0000 7f304ff11640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f304815e6f0 0x7f30480b9640 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.999+0000 7f304ff11640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f302c06c2b0 0x7f302c06e770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.999+0000 7f304ff11640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30480d4f40 0x7f30480a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.999+0000 7f304ff11640 1 --2- 172.21.15.7:0/1866219608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480d1cb0 0x7f30480cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.999+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 >> 172.21.15.7:0/1866219608 conn(0x7f304808bfd0 msgr2=0x7f30480d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.999+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 shutdown_connections 2024-09-15T23:37:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:09.999+0000 7f304ff11640 1 -- 172.21.15.7:0/1866219608 wait complete. 2024-09-15T23:37:10.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-15T23:37:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.123+0000 7f9262c77640 1 Processor -- start 2024-09-15T23:37:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.123+0000 7f9262c77640 1 -- start start 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.124+0000 7f9262c77640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f925c09b610 0x7f925c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.124+0000 7f9262c77640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f925c0ce590 0x7f925c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.124+0000 7f9262c77640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c0d14a0 con 0x7f925c15e870 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.124+0000 7f9262c77640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f925c15ec50 con 0x7f925c09b610 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.124+0000 7f9262c77640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f925c0d1780 con 0x7f925c0ce590 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.124+0000 7f925bfff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f925c09b610 0x7f925c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.124+0000 7f925bfff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f925c09b610 0x7f925c0ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53524/0 (socket says 172.21.15.7:53524) 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.124+0000 7f925bfff640 1 -- 172.21.15.7:0/1995611283 learned_addr learned my addr 172.21.15.7:0/1995611283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.125+0000 7f9261214640 1 --2- 172.21.15.7:0/1995611283 >> v2:172.21.15.18:3300/0 conn(0x7f925c0ce590 0x7f925c06ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.125+0000 7f925bfff640 1 -- 172.21.15.7:0/1995611283 >> v2:172.21.15.18:3300/0 conn(0x7f925c0ce590 msgr2=0x7f925c06ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.125+0000 7f925bfff640 1 --2- 172.21.15.7:0/1995611283 >> v2:172.21.15.18:3300/0 conn(0x7f925c0ce590 0x7f925c06ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.125+0000 7f925bfff640 1 -- 172.21.15.7:0/1995611283 >> v1:172.21.15.18:6789/0 conn(0x7f925c15e870 legacy=0x7f925c0d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.125+0000 7f925bfff640 1 -- 172.21.15.7:0/1995611283 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c0cd850 con 0x7f925c09b610 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.125+0000 7f925bfff640 1 --2- 172.21.15.7:0/1995611283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 0x7f925c0ce050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b4a50 tx=0x7f925002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a714235f9a11f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f925b7fe640 1 -- 172.21.15.7:0/1995611283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250036070 con 0x7f925c09b610 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f925b7fe640 1 -- 172.21.15.7:0/1995611283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9250031600 con 0x7f925c09b610 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f925b7fe640 1 -- 172.21.15.7:0/1995611283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92500319f0 con 0x7f925c09b610 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f9262c77640 1 -- 172.21.15.7:0/1995611283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 msgr2=0x7f925c0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f9262c77640 1 --2- 172.21.15.7:0/1995611283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 0x7f925c0ce050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b4a50 tx=0x7f925002a7c0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f9262c77640 1 -- 172.21.15.7:0/1995611283 shutdown_connections 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f9262c77640 1 --2- 172.21.15.7:0/1995611283 >> v2:172.21.15.18:3300/0 conn(0x7f925c0ce590 0x7f925c06ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f9262c77640 1 --2- 172.21.15.7:0/1995611283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 0x7f925c0ce050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f9262c77640 1 -- 172.21.15.7:0/1995611283 >> 172.21.15.7:0/1995611283 conn(0x7f925c0b3f30 msgr2=0x7f925c0b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f9262c77640 1 -- 172.21.15.7:0/1995611283 shutdown_connections 2024-09-15T23:37:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.126+0000 7f9262c77640 1 -- 172.21.15.7:0/1995611283 wait complete. 2024-09-15T23:37:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.127+0000 7f9262c77640 1 Processor -- start 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.128+0000 7f9262c77640 1 -- start start 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.128+0000 7f9262c77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 0x7f925c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.128+0000 7f9262c77640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f925c0ce590 0x7f925c1f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.128+0000 7f9262c77640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f925c15e870 0x7f925c1f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.128+0000 7f9262c77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f925c06cfa0 con 0x7f925c09b610 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.128+0000 7f9262c77640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0cd550 con 0x7f925c0ce590 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.128+0000 7f9262c77640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f925c06d120 con 0x7f925c15e870 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9260a13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 0x7f925c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9260a13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 0x7f925c15d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53528/0 (socket says 172.21.15.7:53528) 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9260a13640 1 -- 172.21.15.7:0/2577931002 learned_addr learned my addr 172.21.15.7:0/2577931002 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f925bfff640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f925c0ce590 0x7f925c1f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9261214640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f925c15e870 0x7f925c1f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9260a13640 1 -- 172.21.15.7:0/2577931002 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f925c15e870 msgr2=0x7f925c1f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9260a13640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f925c15e870 0x7f925c1f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9260a13640 1 -- 172.21.15.7:0/2577931002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f925c0ce590 msgr2=0x7f925c1f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9260a13640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f925c0ce590 0x7f925c1f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9260a13640 1 -- 172.21.15.7:0/2577931002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c0cb9c0 con 0x7f925c09b610 2024-09-15T23:37:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9260a13640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 0x7f925c15d310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c3c0 tx=0x7f924c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f92597fa640 1 -- 172.21.15.7:0/2577931002 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0052d0 con 0x7f925c09b610 2024-09-15T23:37:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f92597fa640 1 -- 172.21.15.7:0/2577931002 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924c005490 con 0x7f925c09b610 2024-09-15T23:37:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c0cbca0 con 0x7f925c09b610 2024-09-15T23:37:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f92597fa640 1 -- 172.21.15.7:0/2577931002 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c007260 con 0x7f925c09b610 2024-09-15T23:37:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.129+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c15ddb0 con 0x7f925c09b610 2024-09-15T23:37:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.130+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925c0ce050 con 0x7f925c09b610 2024-09-15T23:37:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.131+0000 7f92597fa640 1 -- 172.21.15.7:0/2577931002 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f924c007400 con 0x7f925c09b610 2024-09-15T23:37:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.131+0000 7f92597fa640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f924806c2b0 0x7f924806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.131+0000 7f925bfff640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f924806c2b0 0x7f924806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.131+0000 7f92597fa640 1 -- 172.21.15.7:0/2577931002 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f924c08a6a0 con 0x7f925c09b610 2024-09-15T23:37:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.132+0000 7f925bfff640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f924806c2b0 0x7f924806e770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9250033000 tx=0x7f9250009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:10.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.136+0000 7f92597fa640 1 -- 172.21.15.7:0/2577931002 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f924c058980 con 0x7f925c09b610 2024-09-15T23:37:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.384+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f925c06fa30 con 0x7f925c09b610 2024-09-15T23:37:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.384+0000 7f92597fa640 1 -- 172.21.15.7:0/2577931002 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f924c08a030 con 0x7f925c09b610 2024-09-15T23:37:10.385 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:10.385 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:10.385 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-15T23:37:10.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.385 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-15T23:37:10.385 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQA/b+dm8sQ4OhAAHNZoIaJ8fc1QsnNbpp/DaA==", 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBAb+dmCguhChAAfPnuWwKoFi6kV10uKpCwdg==", 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-15T23:37:10.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBAb+dmr3DPBRAAUbe2RHFM8HJDg4BRIDAudQ==", 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBAb+dmwj7cERAALKBbhukiYUuNyT2l6wb6Qg==", 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBBb+dmCQ0fNhAAtZA4gUeW4XMqjUowJWFlfw==", 2024-09-15T23:37:10.388 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBFb+dmOIk9DBAAgRefS5wwcECApTOBRXFspA==", 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBIb+dmUwJDDhAAUavZVwET5KkoWqm7EgEyCA==", 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBLb+dmI8B4EBAAd1BtncP93hTVHCSHA8tOsQ==", 2024-09-15T23:37:10.390 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBQb+dmiUZxARAAyQdLywX3G5pqXWdrtFwx0A==", 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBTb+dmd/ZCBBAAM7eouMiF2LtaRO8EIL/yXw==", 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.392 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBWb+dm2SLSBhAA6PQkqgVM1UqzW4GJFY5Wmw==", 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBZb+dmktF2JhAA3PMlViiCgsEKbfcuHgaohw==", 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBAb+dmllwaFhAA/nHwGTNRbiagVIKXM0AeYA==", 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBAb+dmFjkbHxAARK/PGgkfecGx1az8oVeB/A==", 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBAb+dmn1LJGxAAaqaLtTaYKCaYfAK1JDdPcw==", 2024-09-15T23:37:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBAb+dm8wjKJBAAlegpp7WS85d0LtyW2elsiw==", 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-15T23:37:10.396 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQA/b+dmu41LHhAADgJ93Ytk00A9nbHm//0kCg==", 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mds", 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBfb+dmP6llIhAAr5y2RZMBuk4t6gcm7ez/cQ==", 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mds" 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mgr", 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBfb+dm8bFlIhAAEvqxXanBdcPbb+zU9PpG2g==", 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-osd", 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBfb+dmFrplIhAAEDF2q3VM3uRNl7FfPN3gmg==", 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-osd" 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd", 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBfb+dmbMFlIhAAx/lfXIzfcoEJcqEftacSaA==", 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBfb+dmBsllIhAAThYE0ZaFowD1WZyzn+7tWA==", 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rgw", 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBfb+dmwNBlIhAA0Ds4BG0NbbBKCc/CtTQKKQ==", 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-15T23:37:10.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQA/b+dmLqHhNhAA2urEMHBpz0vf+Y1IoyfwDw==", 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQA/b+dmb95IMxAAZ5cqDFQOkeXaL/53/u/9HA==", 2024-09-15T23:37:10.401 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-15T23:37:10.402 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-15T23:37:10.402 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-15T23:37:10.402 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-15T23:37:10.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:10.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-15T23:37:10.402 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:10.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f924806c2b0 msgr2=0x7f924806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f924806c2b0 0x7f924806e770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9250033000 tx=0x7f9250009000 comp rx=0 tx=0).stop 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 msgr2=0x7f925c15d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 0x7f925c15d310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c3c0 tx=0x7f924c00c880 comp rx=0 tx=0).stop 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 shutdown_connections 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f924806c2b0 0x7f924806e770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f925c15e870 0x7f925c1f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f925c0ce590 0x7f925c1f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 --2- 172.21.15.7:0/2577931002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f925c09b610 0x7f925c15d310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.386+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 >> 172.21.15.7:0/2577931002 conn(0x7f925c0b3f30 msgr2=0x7f925c09ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.387+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 shutdown_connections 2024-09-15T23:37:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.387+0000 7f9262c77640 1 -- 172.21.15.7:0/2577931002 wait complete. 2024-09-15T23:37:10.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-15T23:37:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.507+0000 7fbd0065e640 1 Processor -- start 2024-09-15T23:37:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.507+0000 7fbd0065e640 1 -- start start 2024-09-15T23:37:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbd0065e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbcf815e6f0 0x7fbcf80d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbd0065e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf80d4f40 0x7fbcf80d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbd0065e640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf80d2ef0 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbd0065e640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf80d31d0 con 0x7fbcf815e6f0 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbd0065e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf80d34b0 con 0x7fbcf80d4f40 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbcfe3fa640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbcf815e6f0 0x7fbcf80d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbcfebfb640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbcf80d1cb0 0x7fbcf80cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49940/0 (socket says 172.21.15.7:49940) 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbcfebfb640 1 -- 172.21.15.7:0/928173813 learned_addr learned my addr 172.21.15.7:0/928173813 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbcfdbf9640 1 --2- 172.21.15.7:0/928173813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf80d4f40 0x7fbcf80d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbcfe3fa640 1 -- 172.21.15.7:0/928173813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf80d4f40 msgr2=0x7fbcf80d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbcfe3fa640 1 --2- 172.21.15.7:0/928173813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf80d4f40 0x7fbcf80d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.508+0000 7fbcfe3fa640 1 -- 172.21.15.7:0/928173813 >> v1:172.21.15.7:6789/0 conn(0x7fbcf80d1cb0 legacy=0x7fbcf80cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.509+0000 7fbcfe3fa640 1 -- 172.21.15.7:0/928173813 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf806ce60 con 0x7fbcf815e6f0 2024-09-15T23:37:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.509+0000 7fbcfd3f8640 1 -- 172.21.15.7:0/928173813 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2337395724 0 0) 0x7fbcf80d2ef0 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.509+0000 7fbcfe3fa640 1 --2- 172.21.15.7:0/928173813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf815e6f0 0x7fbcf80d0e60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40097a0 tx=0x7fbcf402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935d2013c057e47c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:10.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.509+0000 7fbcfd3f8640 1 -- 172.21.15.7:0/928173813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4037070 con 0x7fbcf815e6f0 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.509+0000 7fbcfd3f8640 1 -- 172.21.15.7:0/928173813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf4032440 con 0x7fbcf815e6f0 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.509+0000 7fbcfd3f8640 1 -- 172.21.15.7:0/928173813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4032810 con 0x7fbcf815e6f0 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.510+0000 7fbd0065e640 1 -- 172.21.15.7:0/928173813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf815e6f0 msgr2=0x7fbcf80d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.510+0000 7fbd0065e640 1 --2- 172.21.15.7:0/928173813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf815e6f0 0x7fbcf80d0e60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40097a0 tx=0x7fbcf402a180 comp rx=0 tx=0).stop 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.510+0000 7fbd0065e640 1 -- 172.21.15.7:0/928173813 shutdown_connections 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.510+0000 7fbd0065e640 1 --2- 172.21.15.7:0/928173813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf80d4f40 0x7fbcf80d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.510+0000 7fbd0065e640 1 --2- 172.21.15.7:0/928173813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf815e6f0 0x7fbcf80d0e60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.510+0000 7fbd0065e640 1 -- 172.21.15.7:0/928173813 >> 172.21.15.7:0/928173813 conn(0x7fbcf808bfd0 msgr2=0x7fbcf80d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.510+0000 7fbd0065e640 1 -- 172.21.15.7:0/928173813 shutdown_connections 2024-09-15T23:37:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.510+0000 7fbd0065e640 1 -- 172.21.15.7:0/928173813 wait complete. 2024-09-15T23:37:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.511+0000 7fbd0065e640 1 Processor -- start 2024-09-15T23:37:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.511+0000 7fbd0065e640 1 -- start start 2024-09-15T23:37:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbd0065e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf80d1cb0 0x7fbcf80ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbd0065e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf80d4f40 0x7fbcf814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbd0065e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf815e6f0 0x7fbcf80b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbd0065e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf815ead0 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbd0065e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf80d2be0 con 0x7fbcf80d4f40 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbd0065e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf815ec50 con 0x7fbcf815e6f0 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfe3fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf80d1cb0 0x7fbcf80ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfe3fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf80d1cb0 0x7fbcf80ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53538/0 (socket says 172.21.15.7:53538) 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfe3fa640 1 -- 172.21.15.7:0/2725698354 learned_addr learned my addr 172.21.15.7:0/2725698354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfebfb640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf815e6f0 0x7fbcf80b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfdbf9640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf80d4f40 0x7fbcf814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfe3fa640 1 -- 172.21.15.7:0/2725698354 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf815e6f0 msgr2=0x7fbcf80b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfe3fa640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf815e6f0 0x7fbcf80b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfe3fa640 1 -- 172.21.15.7:0/2725698354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf80d4f40 msgr2=0x7fbcf814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfe3fa640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf80d4f40 0x7fbcf814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.512+0000 7fbcfe3fa640 1 -- 172.21.15.7:0/2725698354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf80b8df0 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.513+0000 7fbcfe3fa640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf80d1cb0 0x7fbcf80ccdd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40098c0 tx=0x7fbcf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.513+0000 7fbceeffd640 1 -- 172.21.15.7:0/2725698354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4041070 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.513+0000 7fbceeffd640 1 -- 172.21.15.7:0/2725698354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf403c930 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.513+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf80cd890 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.513+0000 7fbceeffd640 1 -- 172.21.15.7:0/2725698354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4037040 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.513+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf80cde50 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.514+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf80d0e60 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.514+0000 7fbceeffd640 1 -- 172.21.15.7:0/2725698354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbcf4044070 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.515+0000 7fbceeffd640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcdc06c2b0 0x7fbcdc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.515+0000 7fbceeffd640 1 -- 172.21.15.7:0/2725698354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fbcf40abe20 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.515+0000 7fbcfdbf9640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcdc06c2b0 0x7fbcdc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.515+0000 7fbcfdbf9640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcdc06c2b0 0x7fbcdc06e770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbcf815d180 tx=0x7fbce8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:10.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.519+0000 7fbceeffd640 1 -- 172.21.15.7:0/2725698354 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbcf407a900 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.761+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fbcf80cfdc0 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.763+0000 7fbceeffd640 1 -- 172.21.15.7:0/2725698354 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fbcf407a200 con 0x7fbcf80d1cb0 2024-09-15T23:37:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcdc06c2b0 msgr2=0x7fbcdc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcdc06c2b0 0x7fbcdc06e770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbcf815d180 tx=0x7fbce8008040 comp rx=0 tx=0).stop 2024-09-15T23:37:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf80d1cb0 msgr2=0x7fbcf80ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf80d1cb0 0x7fbcf80ccdd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40098c0 tx=0x7fbcf40046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 shutdown_connections 2024-09-15T23:37:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fbcdc06c2b0 0x7fbcdc06e770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbcf815e6f0 0x7fbcf80b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbcf80d4f40 0x7fbcf814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 --2- 172.21.15.7:0/2725698354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf80d1cb0 0x7fbcf80ccdd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 >> 172.21.15.7:0/2725698354 conn(0x7fbcf808bfd0 msgr2=0x7fbcf80d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 shutdown_connections 2024-09-15T23:37:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.766+0000 7fbd0065e640 1 -- 172.21.15.7:0/2725698354 wait complete. 2024-09-15T23:37:10.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-15T23:37:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.880+0000 7f9fc3cdd640 1 Processor -- start 2024-09-15T23:37:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.880+0000 7f9fc3cdd640 1 -- start start 2024-09-15T23:37:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.881+0000 7f9fc3cdd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 0x7f9fbc0ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.881+0000 7f9fc3cdd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fbc06e060 0x7f9fbc15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.881+0000 7f9fc3cdd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9fbc15f430 0x7f9fbc15b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.881+0000 7f9fc3cdd640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9fbc15f8a0 con 0x7f9fbc06e060 2024-09-15T23:37:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.881+0000 7f9fc3cdd640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f9fbc0b3160 con 0x7f9fbc15f430 2024-09-15T23:37:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.881+0000 7f9fc3cdd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f9fbc0af550 con 0x7f9fbc1558e0 2024-09-15T23:37:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.881+0000 7f9fc1278640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fbc06e060 0x7f9fbc15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.881+0000 7f9fc1278640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fbc06e060 0x7f9fbc15eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53542/0 (socket says 172.21.15.7:53542) 2024-09-15T23:37:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.881+0000 7f9fc1278640 1 -- 172.21.15.7:0/166981290 learned_addr learned my addr 172.21.15.7:0/166981290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.882+0000 7f9fc227a640 1 --2- 172.21.15.7:0/166981290 >> v2:172.21.15.18:3300/0 conn(0x7f9fbc15f430 0x7f9fbc15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.882+0000 7f9fc1a79640 1 --2- 172.21.15.7:0/166981290 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 0x7f9fbc0ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.882+0000 7f9fc1278640 1 -- 172.21.15.7:0/166981290 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 msgr2=0x7f9fbc0ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.882+0000 7f9fc1278640 1 --2- 172.21.15.7:0/166981290 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 0x7f9fbc0ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.882+0000 7f9fc1278640 1 -- 172.21.15.7:0/166981290 >> v2:172.21.15.18:3300/0 conn(0x7f9fbc15f430 msgr2=0x7f9fbc15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.882+0000 7f9fc1278640 1 --2- 172.21.15.7:0/166981290 >> v2:172.21.15.18:3300/0 conn(0x7f9fbc15f430 0x7f9fbc15b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.882+0000 7f9fc1278640 1 -- 172.21.15.7:0/166981290 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc0af270 con 0x7f9fbc06e060 2024-09-15T23:37:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.882+0000 7f9fc227a640 1 --2- 172.21.15.7:0/166981290 >> v2:172.21.15.18:3300/0 conn(0x7f9fbc15f430 0x7f9fbc15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.882+0000 7f9fc1278640 1 --2- 172.21.15.7:0/166981290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 0x7f9fbc15eef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc06dbb0 tx=0x7f9fac02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fcfa350f2a2cff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc0a77640 1 -- 172.21.15.7:0/166981290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac037070 con 0x7f9fbc06e060 2024-09-15T23:37:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc0a77640 1 -- 172.21.15.7:0/166981290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fac02ae00 con 0x7f9fbc06e060 2024-09-15T23:37:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc0a77640 1 -- 172.21.15.7:0/166981290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac032660 con 0x7f9fbc06e060 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/166981290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 msgr2=0x7f9fbc15eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/166981290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 0x7f9fbc15eef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc06dbb0 tx=0x7f9fac02a0b0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/166981290 shutdown_connections 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/166981290 >> v2:172.21.15.18:3300/0 conn(0x7f9fbc15f430 0x7f9fbc15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/166981290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 0x7f9fbc15eef0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/166981290 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 0x7f9fbc0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/166981290 >> 172.21.15.7:0/166981290 conn(0x7f9fbc0b1e60 msgr2=0x7f9fbc0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/166981290 shutdown_connections 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.883+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/166981290 wait complete. 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.884+0000 7f9fc3cdd640 1 Processor -- start 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc3cdd640 1 -- start start 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc3cdd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 0x7f9fbc09de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc3cdd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 0x7f9fbc09a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc3cdd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fbc15f430 0x7f9fbc151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc3cdd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc0aeef0 con 0x7f9fbc06e060 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc3cdd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc072300 con 0x7f9fbc15f430 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc3cdd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f9fbc155cc0 con 0x7f9fbc1558e0 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc1a79640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 0x7f9fbc09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc1a79640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 0x7f9fbc09de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53552/0 (socket says 172.21.15.7:53552) 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc1a79640 1 -- 172.21.15.7:0/1354083755 learned_addr learned my addr 172.21.15.7:0/1354083755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.885+0000 7f9fc227a640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fbc15f430 0x7f9fbc151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fc1278640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 0x7f9fbc09a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fc1a79640 1 -- 172.21.15.7:0/1354083755 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 msgr2=0x7f9fbc09a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fc1a79640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 0x7f9fbc09a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fc1a79640 1 -- 172.21.15.7:0/1354083755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fbc15f430 msgr2=0x7f9fbc151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fc1a79640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fbc15f430 0x7f9fbc151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fc1a79640 1 -- 172.21.15.7:0/1354083755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc09cdf0 con 0x7f9fbc06e060 2024-09-15T23:37:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fc1a79640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 0x7f9fbc09de50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8007e90 tx=0x7f9fb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fb27fc640 1 -- 172.21.15.7:0/1354083755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8014420 con 0x7f9fbc06e060 2024-09-15T23:37:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fb27fc640 1 -- 172.21.15.7:0/1354083755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb8019070 con 0x7f9fbc06e060 2024-09-15T23:37:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fbc0cfb50 con 0x7f9fbc06e060 2024-09-15T23:37:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fb27fc640 1 -- 172.21.15.7:0/1354083755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8014800 con 0x7f9fbc06e060 2024-09-15T23:37:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.886+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fbc09d370 con 0x7f9fbc06e060 2024-09-15T23:37:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.887+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fbc15eef0 con 0x7f9fbc06e060 2024-09-15T23:37:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.888+0000 7f9fb27fc640 1 -- 172.21.15.7:0/1354083755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9fb80149a0 con 0x7f9fbc06e060 2024-09-15T23:37:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.888+0000 7f9fb27fc640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9fa006c2b0 0x7f9fa006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.888+0000 7f9fb27fc640 1 -- 172.21.15.7:0/1354083755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f9fb808b8a0 con 0x7f9fbc06e060 2024-09-15T23:37:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.888+0000 7f9fc1278640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9fa006c2b0 0x7f9fa006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.889+0000 7f9fc1278640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9fa006c2b0 0x7f9fa006e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9fac034000 tx=0x7f9fac02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:10.893+0000 7f9fb27fc640 1 -- 172.21.15.7:0/1354083755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9fb8059c10 con 0x7f9fbc06e060 2024-09-15T23:37:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.155+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f9fbc0b2d10 con 0x7f9fbc06e060 2024-09-15T23:37:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.157+0000 7f9fb27fc640 1 -- 172.21.15.7:0/1354083755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f9fb805df10 con 0x7f9fbc06e060 2024-09-15T23:37:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9fa006c2b0 msgr2=0x7f9fa006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9fa006c2b0 0x7f9fa006e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9fac034000 tx=0x7f9fac02e000 comp rx=0 tx=0).stop 2024-09-15T23:37:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 msgr2=0x7f9fbc09de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 0x7f9fbc09de50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8007e90 tx=0x7f9fb800e4b0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 shutdown_connections 2024-09-15T23:37:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fbc15f430 0x7f9fbc151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9fa006c2b0 0x7f9fa006e770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9fbc1558e0 0x7f9fbc09a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 --2- 172.21.15.7:0/1354083755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fbc06e060 0x7f9fbc09de50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 >> 172.21.15.7:0/1354083755 conn(0x7f9fbc0b1e60 msgr2=0x7f9fbc15a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 shutdown_connections 2024-09-15T23:37:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.159+0000 7f9fc3cdd640 1 -- 172.21.15.7:0/1354083755 wait complete. 2024-09-15T23:37:11.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-15T23:37:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.274+0000 7f9cc9526640 1 Processor -- start 2024-09-15T23:37:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.274+0000 7f9cc9526640 1 -- start start 2024-09-15T23:37:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.274+0000 7f9cc9526640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9cc40af370 0x7f9cc406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.274+0000 7f9cc9526640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc40cb210 con 0x7f9cc406bc10 2024-09-15T23:37:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.274+0000 7f9cc9526640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc40701a0 con 0x7f9cc40cadb0 2024-09-15T23:37:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.274+0000 7f9cc9526640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc406f570 con 0x7f9cc40af370 2024-09-15T23:37:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc2ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9cc406bc10 0x7f9cc406fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49952/0 (socket says 172.21.15.7:49952) 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc2ffd640 1 -- 172.21.15.7:0/2990819901 learned_addr learned my addr 172.21.15.7:0/2990819901 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc37fe640 1 --2- 172.21.15.7:0/2990819901 >> v2:172.21.15.18:3300/0 conn(0x7f9cc40af370 0x7f9cc406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337750593 0 0) 0x7f9cc40cb210 con 0x7f9cc406bc10 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cac003520 con 0x7f9cc406bc10 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2323959015 0 0) 0x7f9cac003520 con 0x7f9cc406bc10 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 >> v2:172.21.15.18:3300/0 conn(0x7f9cc40af370 msgr2=0x7f9cc406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 --2- 172.21.15.7:0/2990819901 >> v2:172.21.15.18:3300/0 conn(0x7f9cc40af370 0x7f9cc406bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 >> v1:172.21.15.18:6789/0 conn(0x7f9cc40cadb0 legacy=0x7f9cc40aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc406f290 con 0x7f9cc406bc10 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f9cb4001670 con 0x7f9cc406bc10 2024-09-15T23:37:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 >> v1:172.21.15.7:6789/0 conn(0x7f9cc406bc10 legacy=0x7f9cc406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac0039b0 0x7f9cac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9cac004800 0x7f9cac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac0051c0 0x7f9cac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac00c030 con 0x7f9cac0039b0 2024-09-15T23:37:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac00c1e0 con 0x7f9cac0051c0 2024-09-15T23:37:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac00c3e0 con 0x7f9cac004800 2024-09-15T23:37:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc2ffd640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac0039b0 0x7f9cac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc37fe640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac0051c0 0x7f9cac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.275+0000 7f9cc27fc640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9cac004800 0x7f9cac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc2ffd640 1 -- 172.21.15.7:0/2990819901 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9cac004800 msgr2=0x7f9cac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:11.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc2ffd640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9cac004800 0x7f9cac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc2ffd640 1 -- 172.21.15.7:0/2990819901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac0051c0 msgr2=0x7f9cac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:11.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc2ffd640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac0051c0 0x7f9cac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc2ffd640 1 -- 172.21.15.7:0/2990819901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac003520 con 0x7f9cac0039b0 2024-09-15T23:37:11.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc37fe640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac0051c0 0x7f9cac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:11.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc37fe640 1 -- 172.21.15.7:0/2990819901 reap_dead start 2024-09-15T23:37:11.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc2ffd640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac0039b0 0x7f9cac003dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400c210 tx=0x7f9cac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951fbf1ed9fc8078 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4007d30 con 0x7f9cac0039b0 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb40058d0 con 0x7f9cac0039b0 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.276+0000 7f9cc1ffb640 1 -- 172.21.15.7:0/2990819901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4005d20 con 0x7f9cac0039b0 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.277+0000 7f9cc9526640 1 -- 172.21.15.7:0/2990819901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac0039b0 msgr2=0x7f9cac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.277+0000 7f9cc9526640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac0039b0 0x7f9cac003dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400c210 tx=0x7f9cac005830 comp rx=0 tx=0).stop 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.277+0000 7f9cc9526640 1 -- 172.21.15.7:0/2990819901 shutdown_connections 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.277+0000 7f9cc9526640 1 --2- 172.21.15.7:0/2990819901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac0039b0 0x7f9cac003dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.277+0000 7f9cc9526640 1 -- 172.21.15.7:0/2990819901 >> 172.21.15.7:0/2990819901 conn(0x7f9cc40b2fc0 msgr2=0x7f9cc40b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.277+0000 7f9cc9526640 1 -- 172.21.15.7:0/2990819901 shutdown_connections 2024-09-15T23:37:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.277+0000 7f9cc9526640 1 -- 172.21.15.7:0/2990819901 wait complete. 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.278+0000 7f9cc9526640 1 Processor -- start 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.278+0000 7f9cc9526640 1 -- start start 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc9526640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9cac0039b0 0x7f9cc415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc9526640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cc4156f40 0x7f9cc41573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc9526640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc4151520 0x7f9cc41519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc9526640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc40ca7e0 con 0x7f9cc4151520 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc9526640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc40cb780 con 0x7f9cc4156f40 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc9526640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc406fa90 con 0x7f9cac0039b0 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc4151520 0x7f9cc41519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc4151520 0x7f9cc41519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53570/0 (socket says 172.21.15.7:53570) 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc37fe640 1 -- 172.21.15.7:0/2982404776 learned_addr learned my addr 172.21.15.7:0/2982404776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc27fc640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cc4156f40 0x7f9cc41573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.279+0000 7f9cc2ffd640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9cac0039b0 0x7f9cc415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cc37fe640 1 -- 172.21.15.7:0/2982404776 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9cac0039b0 msgr2=0x7f9cc415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cc37fe640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9cac0039b0 0x7f9cc415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cc37fe640 1 -- 172.21.15.7:0/2982404776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cc4156f40 msgr2=0x7f9cc41573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cc37fe640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cc4156f40 0x7f9cc41573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cc37fe640 1 -- 172.21.15.7:0/2982404776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc40b5300 con 0x7f9cc4151520 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cc37fe640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc4151520 0x7f9cc41519a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40ced50 tx=0x7f9cbc010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cab7fe640 1 -- 172.21.15.7:0/2982404776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc010dd0 con 0x7f9cc4151520 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cab7fe640 1 -- 172.21.15.7:0/2982404776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cbc003990 con 0x7f9cc4151520 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cab7fe640 1 -- 172.21.15.7:0/2982404776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc003d80 con 0x7f9cc4151520 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc40b5b20 con 0x7f9cc4151520 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.280+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc40b60e0 con 0x7f9cc4151520 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.281+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc40b62e0 con 0x7f9cc4151520 2024-09-15T23:37:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.282+0000 7f9cab7fe640 1 -- 172.21.15.7:0/2982404776 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9cbc005d10 con 0x7f9cc4151520 2024-09-15T23:37:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.282+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9cb006c2e0 0x7f9cb006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.282+0000 7f9cab7fe640 1 -- 172.21.15.7:0/2982404776 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f9cbc08b6c0 con 0x7f9cc4151520 2024-09-15T23:37:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.282+0000 7f9cc2ffd640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9cb006c2e0 0x7f9cb006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.283+0000 7f9cc2ffd640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9cb006c2e0 0x7f9cb006e7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4033000 tx=0x7f9cb402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.287+0000 7f9cab7fe640 1 -- 172.21.15.7:0/2982404776 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9cc40b62e0 con 0x7f9cc4151520 2024-09-15T23:37:11.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.530+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9cc406fe20 con 0x7f9cc4151520 2024-09-15T23:37:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.532+0000 7f9cab7fe640 1 -- 172.21.15.7:0/2982404776 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f9cbc025020 con 0x7f9cc4151520 2024-09-15T23:37:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9cb006c2e0 msgr2=0x7f9cb006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9cb006c2e0 0x7f9cb006e7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4033000 tx=0x7f9cb402e000 comp rx=0 tx=0).stop 2024-09-15T23:37:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc4151520 msgr2=0x7f9cc41519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc4151520 0x7f9cc41519a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40ced50 tx=0x7f9cbc010590 comp rx=0 tx=0).stop 2024-09-15T23:37:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 shutdown_connections 2024-09-15T23:37:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc4151520 0x7f9cc41519a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cc4156f40 0x7f9cc41573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f9cb006c2e0 0x7f9cb006e7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 --2- 172.21.15.7:0/2982404776 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9cac0039b0 0x7f9cc415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 >> 172.21.15.7:0/2982404776 conn(0x7f9cc40b2fc0 msgr2=0x7f9cc40b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 shutdown_connections 2024-09-15T23:37:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.535+0000 7f9cc9526640 1 -- 172.21.15.7:0/2982404776 wait complete. 2024-09-15T23:37:11.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-15T23:37:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.666+0000 7f1eb180a640 1 Processor -- start 2024-09-15T23:37:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.666+0000 7f1eb180a640 1 -- start start 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eb180a640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1eac06e060 0x7f1eac15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eb180a640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eac15f850 con 0x7f1eac15f430 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eb180a640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eac0af270 con 0x7f1eac1558e0 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eb180a640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f1eac0b3160 con 0x7f1eac06e060 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eab7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1eac15f430 0x7f1eac072b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49962/0 (socket says 172.21.15.7:49962) 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eab7fe640 1 -- 172.21.15.7:0/1689845727 learned_addr learned my addr 172.21.15.7:0/1689845727 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1ea27fc640 1 --2- 172.21.15.7:0/1689845727 >> v2:172.21.15.18:3300/0 conn(0x7f1eac06e060 0x7f1eac15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724565868 0 0) 0x7f1eac15f850 con 0x7f1eac15f430 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e98003520 con 0x7f1eac15f430 2024-09-15T23:37:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 586501454 0 0) 0x7f1e98003520 con 0x7f1eac15f430 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 >> v2:172.21.15.18:3300/0 conn(0x7f1eac06e060 msgr2=0x7f1eac15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 --2- 172.21.15.7:0/1689845727 >> v2:172.21.15.18:3300/0 conn(0x7f1eac06e060 0x7f1eac15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 >> v1:172.21.15.18:6789/0 conn(0x7f1eac1558e0 legacy=0x7f1eac0ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac072020 con 0x7f1eac15f430 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f1ea40019f0 con 0x7f1eac15f430 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 >> v1:172.21.15.7:6789/0 conn(0x7f1eac15f430 legacy=0x7f1eac072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e980039b0 0x7f1e98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e98004800 0x7f1e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e980052e0 0x7f1e9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9800c180 con 0x7f1e980052e0 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9800c330 con 0x7f1e98004800 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.667+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9800c530 con 0x7f1e980039b0 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.668+0000 7f1eab7fe640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e980052e0 0x7f1e9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.668+0000 7f1ea27fc640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e98004800 0x7f1e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.668+0000 7f1eaaffd640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e980039b0 0x7f1e98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.668+0000 7f1eab7fe640 1 -- 172.21.15.7:0/1689845727 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e980039b0 msgr2=0x7f1e98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.668+0000 7f1eab7fe640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e980039b0 0x7f1e98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.668+0000 7f1eab7fe640 1 -- 172.21.15.7:0/1689845727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e98004800 msgr2=0x7f1e98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.668+0000 7f1eab7fe640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e98004800 0x7f1e98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.668+0000 7f1eab7fe640 1 -- 172.21.15.7:0/1689845727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e98003520 con 0x7f1e980052e0 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.668+0000 7f1eab7fe640 1 -- 172.21.15.7:0/1689845727 reap_dead start 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eab7fe640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e980052e0 0x7f1e9800b9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400a830 tx=0x7f1e98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de07269dd6d777e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400a480 con 0x7f1e980052e0 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea402f070 con 0x7f1e980052e0 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eaa7fc640 1 -- 172.21.15.7:0/1689845727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea402bc90 con 0x7f1e980052e0 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eb180a640 1 -- 172.21.15.7:0/1689845727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e980052e0 msgr2=0x7f1e9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eb180a640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e980052e0 0x7f1e9800b9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400a830 tx=0x7f1e98005830 comp rx=0 tx=0).stop 2024-09-15T23:37:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eb180a640 1 -- 172.21.15.7:0/1689845727 shutdown_connections 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eb180a640 1 --2- 172.21.15.7:0/1689845727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e980052e0 0x7f1e9800b9c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eb180a640 1 -- 172.21.15.7:0/1689845727 >> 172.21.15.7:0/1689845727 conn(0x7f1eac0b1e60 msgr2=0x7f1eac0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eb180a640 1 -- 172.21.15.7:0/1689845727 shutdown_connections 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.669+0000 7f1eb180a640 1 -- 172.21.15.7:0/1689845727 wait complete. 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.670+0000 7f1eb180a640 1 Processor -- start 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.671+0000 7f1eb180a640 1 -- start start 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.671+0000 7f1eb180a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e980052e0 0x7f1eac151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.671+0000 7f1eb180a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eac14c980 0x7f1eac1f2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.671+0000 7f1eb180a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eac14bc70 0x7f1eac14c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.671+0000 7f1eb180a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac155cc0 con 0x7f1eac14bc70 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.671+0000 7f1eb180a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac0af550 con 0x7f1eac14c980 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.671+0000 7f1eb180a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f1eac155e40 con 0x7f1e980052e0 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eac14bc70 0x7f1eac14c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eac14bc70 0x7f1eac14c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53596/0 (socket says 172.21.15.7:53596) 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eab7fe640 1 -- 172.21.15.7:0/1363738314 learned_addr learned my addr 172.21.15.7:0/1363738314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eaaffd640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e980052e0 0x7f1eac151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1ea27fc640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eac14c980 0x7f1eac1f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eab7fe640 1 -- 172.21.15.7:0/1363738314 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e980052e0 msgr2=0x7f1eac151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eab7fe640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e980052e0 0x7f1eac151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eab7fe640 1 -- 172.21.15.7:0/1363738314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eac14c980 msgr2=0x7f1eac1f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eab7fe640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eac14c980 0x7f1eac1f2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eab7fe640 1 -- 172.21.15.7:0/1363738314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac0b3b70 con 0x7f1eac14bc70 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eab7fe640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eac14bc70 0x7f1eac14c100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400a830 tx=0x7f1eac0721f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1ea8ff9640 1 -- 172.21.15.7:0/1363738314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400a480 con 0x7f1eac14bc70 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1ea8ff9640 1 -- 172.21.15.7:0/1363738314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea402bd90 con 0x7f1eac14bc70 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac152020 con 0x7f1eac14bc70 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1ea8ff9640 1 -- 172.21.15.7:0/1363738314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004d80 con 0x7f1eac14bc70 2024-09-15T23:37:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.672+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac0b4110 con 0x7f1eac14bc70 2024-09-15T23:37:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.673+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eac06f8d0 con 0x7f1eac14bc70 2024-09-15T23:37:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.674+0000 7f1ea8ff9640 1 -- 172.21.15.7:0/1363738314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1ea4034470 con 0x7f1eac14bc70 2024-09-15T23:37:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.674+0000 7f1ea8ff9640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1e8c06c2b0 0x7f1e8c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.674+0000 7f1ea8ff9640 1 -- 172.21.15.7:0/1363738314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f1ea40aad80 con 0x7f1eac14bc70 2024-09-15T23:37:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.675+0000 7f1eaaffd640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1e8c06c2b0 0x7f1e8c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.675+0000 7f1eaaffd640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1e8c06c2b0 0x7f1e8c06e770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000c00 tx=0x7f1e9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.679+0000 7f1ea8ff9640 1 -- 172.21.15.7:0/1363738314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1eac06f8d0 con 0x7f1eac14bc70 2024-09-15T23:37:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.921+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f1eac15eef0 con 0x7f1eac14bc70 2024-09-15T23:37:11.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.923+0000 7f1ea8ff9640 1 -- 172.21.15.7:0/1363738314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f1ea4079060 con 0x7f1eac14bc70 2024-09-15T23:37:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1e8c06c2b0 msgr2=0x7f1e8c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1e8c06c2b0 0x7f1e8c06e770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000c00 tx=0x7f1e9c009040 comp rx=0 tx=0).stop 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eac14bc70 msgr2=0x7f1eac14c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eac14bc70 0x7f1eac14c100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400a830 tx=0x7f1eac0721f0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 shutdown_connections 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eac14bc70 0x7f1eac14c100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eac14c980 0x7f1eac1f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f1e8c06c2b0 0x7f1e8c06e770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 --2- 172.21.15.7:0/1363738314 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e980052e0 0x7f1eac151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 >> 172.21.15.7:0/1363738314 conn(0x7f1eac0b1e60 msgr2=0x7f1eac06e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.925+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 shutdown_connections 2024-09-15T23:37:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:11.926+0000 7f1eb180a640 1 -- 172.21.15.7:0/1363738314 wait complete. 2024-09-15T23:37:11.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-15T23:37:12.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.040+0000 7f0571f68640 1 Processor -- start 2024-09-15T23:37:12.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.041+0000 7f0571f68640 1 -- start start 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.041+0000 7f0571f68640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f056c1609d0 0x7f056c0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.041+0000 7f0571f68640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056c160050 con 0x7f056c0b1be0 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.041+0000 7f0571f68640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056c09bcf0 con 0x7f056c15fc70 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.041+0000 7f0571f68640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f056c0cd620 con 0x7f056c1609d0 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.042+0000 7f056affd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f056c0b1be0 0x7f056c1602c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49976/0 (socket says 172.21.15.7:49976) 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.042+0000 7f056affd640 1 -- 172.21.15.7:0/246512216 learned_addr learned my addr 172.21.15.7:0/246512216 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.042+0000 7f056bfff640 1 --2- 172.21.15.7:0/246512216 >> v2:172.21.15.18:3300/0 conn(0x7f056c1609d0 0x7f056c0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.042+0000 7f056a7fc640 1 -- 172.21.15.7:0/246512216 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2058561130 0 0) 0x7f056c160050 con 0x7f056c0b1be0 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.042+0000 7f056a7fc640 1 -- 172.21.15.7:0/246512216 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0554003520 con 0x7f056c0b1be0 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.042+0000 7f056bfff640 1 -- 172.21.15.7:0/246512216 >> v1:172.21.15.18:6789/0 conn(0x7f056c15fc70 legacy=0x7f056c0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.042+0000 7f056a7fc640 1 -- 172.21.15.7:0/246512216 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3252491040 0 0) 0x7f0554003520 con 0x7f056c0b1be0 2024-09-15T23:37:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.042+0000 7f056bfff640 1 -- 172.21.15.7:0/246512216 >> v1:172.21.15.7:6789/0 conn(0x7f056c0b1be0 legacy=0x7f056c1602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.042+0000 7f056bfff640 1 -- 172.21.15.7:0/246512216 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c09b450 con 0x7f056c1609d0 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.043+0000 7f056bfff640 1 --2- 172.21.15.7:0/246512216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c1609d0 0x7f056c0cdab0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0564009d90 tx=0x7f056402a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457edf2602de40de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.043+0000 7f056a7fc640 1 -- 172.21.15.7:0/246512216 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564035020 con 0x7f056c1609d0 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.043+0000 7f056a7fc640 1 -- 172.21.15.7:0/246512216 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564007540 con 0x7f056c1609d0 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.043+0000 7f056a7fc640 1 -- 172.21.15.7:0/246512216 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560001670 con 0x7f056c1609d0 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.044+0000 7f0571f68640 1 -- 172.21.15.7:0/246512216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c1609d0 msgr2=0x7f056c0cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.044+0000 7f0571f68640 1 --2- 172.21.15.7:0/246512216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c1609d0 0x7f056c0cdab0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0564009d90 tx=0x7f056402a5b0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.044+0000 7f0571f68640 1 -- 172.21.15.7:0/246512216 shutdown_connections 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.044+0000 7f0571f68640 1 --2- 172.21.15.7:0/246512216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c1609d0 0x7f056c0cdab0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.044+0000 7f0571f68640 1 -- 172.21.15.7:0/246512216 >> 172.21.15.7:0/246512216 conn(0x7f056c0b3f40 msgr2=0x7f056c0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.044+0000 7f0571f68640 1 -- 172.21.15.7:0/246512216 shutdown_connections 2024-09-15T23:37:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.044+0000 7f0571f68640 1 -- 172.21.15.7:0/246512216 wait complete. 2024-09-15T23:37:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.045+0000 7f0571f68640 1 Processor -- start 2024-09-15T23:37:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.045+0000 7f0571f68640 1 -- start start 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f0571f68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f056c0b1be0 0x7f056c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f0571f68640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c15fc70 0x7f056c0b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f0571f68640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f056c1609d0 0x7f056c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f0571f68640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f056c0cd2b0 con 0x7f056c0b1be0 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f0571f68640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f056c09ba10 con 0x7f056c15fc70 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f0571f68640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f056c160fb0 con 0x7f056c1609d0 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f056c0b1be0 0x7f056c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f056c0b1be0 0x7f056c153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53606/0 (socket says 172.21.15.7:53606) 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056b7fe640 1 -- 172.21.15.7:0/265120486 learned_addr learned my addr 172.21.15.7:0/265120486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056affd640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c15fc70 0x7f056c0b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056bfff640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f056c1609d0 0x7f056c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056b7fe640 1 -- 172.21.15.7:0/265120486 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f056c1609d0 msgr2=0x7f056c0cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056b7fe640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f056c1609d0 0x7f056c0cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056b7fe640 1 -- 172.21.15.7:0/265120486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c15fc70 msgr2=0x7f056c0b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056b7fe640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c15fc70 0x7f056c0b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056b7fe640 1 -- 172.21.15.7:0/265120486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c0afcc0 con 0x7f056c0b1be0 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056affd640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c15fc70 0x7f056c0b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f056b7fe640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f056c0b1be0 0x7f056c153d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c580 tx=0x7f055c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.046+0000 7f0553fff640 1 -- 172.21.15.7:0/265120486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c017070 con 0x7f056c0b1be0 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.047+0000 7f0553fff640 1 -- 172.21.15.7:0/265120486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055c0035f0 con 0x7f056c0b1be0 2024-09-15T23:37:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.047+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056c0affa0 con 0x7f056c0b1be0 2024-09-15T23:37:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.047+0000 7f0553fff640 1 -- 172.21.15.7:0/265120486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0039e0 con 0x7f056c0b1be0 2024-09-15T23:37:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.047+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056c0b0560 con 0x7f056c0b1be0 2024-09-15T23:37:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.048+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056c0b83d0 con 0x7f056c0b1be0 2024-09-15T23:37:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.048+0000 7f0553fff640 1 -- 172.21.15.7:0/265120486 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f055c01a070 con 0x7f056c0b1be0 2024-09-15T23:37:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.049+0000 7f0553fff640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f055806c2e0 0x7f055806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.049+0000 7f056affd640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f055806c2e0 0x7f055806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.049+0000 7f0553fff640 1 -- 172.21.15.7:0/265120486 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f055c08a680 con 0x7f056c0b1be0 2024-09-15T23:37:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.049+0000 7f056affd640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f055806c2e0 0x7f055806e7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0560001eb0 tx=0x7f0560001c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.053+0000 7f0553fff640 1 -- 172.21.15.7:0/265120486 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f055c058fc0 con 0x7f056c0b1be0 2024-09-15T23:37:12.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.293+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f056c0b30e0 con 0x7f056c0b1be0 2024-09-15T23:37:12.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.295+0000 7f0553fff640 1 -- 172.21.15.7:0/265120486 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f055c0588c0 con 0x7f056c0b1be0 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f055806c2e0 msgr2=0x7f055806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f055806c2e0 0x7f055806e7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0560001eb0 tx=0x7f0560001c40 comp rx=0 tx=0).stop 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f056c0b1be0 msgr2=0x7f056c153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f056c0b1be0 0x7f056c153d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c580 tx=0x7f055c00ca40 comp rx=0 tx=0).stop 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 shutdown_connections 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f055806c2e0 0x7f055806e7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f056c1609d0 0x7f056c0cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c15fc70 0x7f056c0b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 --2- 172.21.15.7:0/265120486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f056c0b1be0 0x7f056c153d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 >> 172.21.15.7:0/265120486 conn(0x7f056c0b3f40 msgr2=0x7f056c0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 shutdown_connections 2024-09-15T23:37:12.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.297+0000 7f0571f68640 1 -- 172.21.15.7:0/265120486 wait complete. 2024-09-15T23:37:12.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-15T23:37:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.411+0000 7f62f03e4640 1 Processor -- start 2024-09-15T23:37:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.411+0000 7f62f03e4640 1 -- start start 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.412+0000 7f62f03e4640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62e815e810 0x7f62e80d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.412+0000 7f62f03e4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f62e80d32c0 0x7f62e80cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.412+0000 7f62f03e4640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e80d5240 con 0x7f62e80d4d10 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.412+0000 7f62f03e4640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f62e80d2a00 con 0x7f62e815e810 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.412+0000 7f62f03e4640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f62e80d23b0 con 0x7f62e80d32c0 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.412+0000 7f62ee180640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62e815e810 0x7f62e80d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.412+0000 7f62ee180640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62e815e810 0x7f62e80d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53608/0 (socket says 172.21.15.7:53608) 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.412+0000 7f62ee180640 1 -- 172.21.15.7:0/813321174 learned_addr learned my addr 172.21.15.7:0/813321174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.413+0000 7f62ee981640 1 --2- 172.21.15.7:0/813321174 >> v2:172.21.15.18:3300/0 conn(0x7f62e80d32c0 0x7f62e80cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.413+0000 7f62ee180640 1 -- 172.21.15.7:0/813321174 >> v2:172.21.15.18:3300/0 conn(0x7f62e80d32c0 msgr2=0x7f62e80cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.413+0000 7f62ee180640 1 --2- 172.21.15.7:0/813321174 >> v2:172.21.15.18:3300/0 conn(0x7f62e80d32c0 0x7f62e80cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.413+0000 7f62ee180640 1 -- 172.21.15.7:0/813321174 >> v1:172.21.15.18:6789/0 conn(0x7f62e80d4d10 legacy=0x7f62e80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.413+0000 7f62ee180640 1 -- 172.21.15.7:0/813321174 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e80d2690 con 0x7f62e815e810 2024-09-15T23:37:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.413+0000 7f62ee180640 1 --2- 172.21.15.7:0/813321174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e815e810 0x7f62e80d17a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f62e808daa0 tx=0x7f62d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18737d52aa840dba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62ed17e640 1 -- 172.21.15.7:0/813321174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8036070 con 0x7f62e815e810 2024-09-15T23:37:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62ed17e640 1 -- 172.21.15.7:0/813321174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d8031600 con 0x7f62e815e810 2024-09-15T23:37:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62ed17e640 1 -- 172.21.15.7:0/813321174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d80319d0 con 0x7f62e815e810 2024-09-15T23:37:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62f03e4640 1 -- 172.21.15.7:0/813321174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e815e810 msgr2=0x7f62e80d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62f03e4640 1 --2- 172.21.15.7:0/813321174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e815e810 0x7f62e80d17a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f62e808daa0 tx=0x7f62d802a6a0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62f03e4640 1 -- 172.21.15.7:0/813321174 shutdown_connections 2024-09-15T23:37:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62f03e4640 1 --2- 172.21.15.7:0/813321174 >> v2:172.21.15.18:3300/0 conn(0x7f62e80d32c0 0x7f62e80cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62f03e4640 1 --2- 172.21.15.7:0/813321174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e815e810 0x7f62e80d17a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62f03e4640 1 -- 172.21.15.7:0/813321174 >> 172.21.15.7:0/813321174 conn(0x7f62e808bfd0 msgr2=0x7f62e80d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62f03e4640 1 -- 172.21.15.7:0/813321174 shutdown_connections 2024-09-15T23:37:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.414+0000 7f62f03e4640 1 -- 172.21.15.7:0/813321174 wait complete. 2024-09-15T23:37:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62f03e4640 1 Processor -- start 2024-09-15T23:37:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62f03e4640 1 -- start start 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62f03e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e80d32c0 0x7f62e8157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62f03e4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62e80d4d10 0x7f62e815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62f03e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62e815e810 0x7f62e815f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62f03e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62e806cec0 con 0x7f62e80d32c0 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62f03e4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f62e806cd40 con 0x7f62e815e810 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62f03e4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f62e80cfbe0 con 0x7f62e80d4d10 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62ee180640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e80d32c0 0x7f62e8157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62ee180640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e80d32c0 0x7f62e8157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53616/0 (socket says 172.21.15.7:53616) 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.416+0000 7f62ee180640 1 -- 172.21.15.7:0/3042241608 learned_addr learned my addr 172.21.15.7:0/3042241608 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62ee981640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62e815e810 0x7f62e815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62ed97f640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62e80d4d10 0x7f62e815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62ee180640 1 -- 172.21.15.7:0/3042241608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62e80d4d10 msgr2=0x7f62e815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62ee180640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62e80d4d10 0x7f62e815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62ee180640 1 -- 172.21.15.7:0/3042241608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62e815e810 msgr2=0x7f62e815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62ee180640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62e815e810 0x7f62e815f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62ee180640 1 -- 172.21.15.7:0/3042241608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e80b9680 con 0x7f62e80d32c0 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62ee180640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e80d32c0 0x7f62e8157ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f62d8034040 tx=0x7f62d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62e6ffd640 1 -- 172.21.15.7:0/3042241608 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8036050 con 0x7f62e80d32c0 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62e6ffd640 1 -- 172.21.15.7:0/3042241608 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d8042070 con 0x7f62e80d32c0 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e80cb920 con 0x7f62e80d32c0 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62e6ffd640 1 -- 172.21.15.7:0/3042241608 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8009510 con 0x7f62e80d32c0 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.417+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e8158320 con 0x7f62e80d32c0 2024-09-15T23:37:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.418+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e80d1d90 con 0x7f62e80d32c0 2024-09-15T23:37:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.419+0000 7f62e6ffd640 1 -- 172.21.15.7:0/3042241608 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f62d8007c50 con 0x7f62e80d32c0 2024-09-15T23:37:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.419+0000 7f62e6ffd640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f62d406c2b0 0x7f62d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.419+0000 7f62e6ffd640 1 -- 172.21.15.7:0/3042241608 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f62d80ac1a0 con 0x7f62e80d32c0 2024-09-15T23:37:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.419+0000 7f62ed97f640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f62d406c2b0 0x7f62d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.420+0000 7f62ed97f640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f62d406c2b0 0x7f62d406e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f62e80b1ee0 tx=0x7f62dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.423+0000 7f62e6ffd640 1 -- 172.21.15.7:0/3042241608 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f62d807ac20 con 0x7f62e80d32c0 2024-09-15T23:37:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.664+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f62e806ea30 con 0x7f62e80d32c0 2024-09-15T23:37:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.666+0000 7f62e6ffd640 1 -- 172.21.15.7:0/3042241608 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f62d807a520 con 0x7f62e80d32c0 2024-09-15T23:37:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f62d406c2b0 msgr2=0x7f62d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f62d406c2b0 0x7f62d406e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f62e80b1ee0 tx=0x7f62dc009040 comp rx=0 tx=0).stop 2024-09-15T23:37:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e80d32c0 msgr2=0x7f62e8157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e80d32c0 0x7f62e8157ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f62d8034040 tx=0x7f62d8004a40 comp rx=0 tx=0).stop 2024-09-15T23:37:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 shutdown_connections 2024-09-15T23:37:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62e815e810 0x7f62e815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f62d406c2b0 0x7f62d406e770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62e80d4d10 0x7f62e815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 --2- 172.21.15.7:0/3042241608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e80d32c0 0x7f62e8157ad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 >> 172.21.15.7:0/3042241608 conn(0x7f62e808bfd0 msgr2=0x7f62e80b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 shutdown_connections 2024-09-15T23:37:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.668+0000 7f62f03e4640 1 -- 172.21.15.7:0/3042241608 wait complete. 2024-09-15T23:37:12.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-15T23:37:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.781+0000 7f8035b3b640 1 Processor -- start 2024-09-15T23:37:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.781+0000 7f8035b3b640 1 -- start start 2024-09-15T23:37:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.782+0000 7f8035b3b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80300d3250 0x7f80300d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.782+0000 7f8035b3b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80300d0360 con 0x7f8030161e60 2024-09-15T23:37:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.782+0000 7f8035b3b640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80300d05d0 con 0x7f80300cfed0 2024-09-15T23:37:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.782+0000 7f8035b3b640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f80300d5000 con 0x7f80300d3250 2024-09-15T23:37:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.782+0000 7f802effd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80300d3250 0x7f80300d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.782+0000 7f802f7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8030161e60 0x7f80300d2b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49984/0 (socket says 172.21.15.7:49984) 2024-09-15T23:37:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.782+0000 7f802f7fe640 1 -- 172.21.15.7:0/2508839855 learned_addr learned my addr 172.21.15.7:0/2508839855 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.783+0000 7f802effd640 1 -- 172.21.15.7:0/2508839855 >> v1:172.21.15.18:6789/0 conn(0x7f80300cfed0 legacy=0x7f80300b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.783+0000 7f802effd640 1 -- 172.21.15.7:0/2508839855 >> v1:172.21.15.7:6789/0 conn(0x7f8030161e60 legacy=0x7f80300d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.783+0000 7f802effd640 1 -- 172.21.15.7:0/2508839855 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80300d08b0 con 0x7f80300d3250 2024-09-15T23:37:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.783+0000 7f802effd640 1 --2- 172.21.15.7:0/2508839855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80300d3250 0x7f80300d51a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8024009790 tx=0x7f802402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8bf8e200d34eea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.783+0000 7f802e7fc640 1 -- 172.21.15.7:0/2508839855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024037070 con 0x7f80300d3250 2024-09-15T23:37:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.784+0000 7f802e7fc640 1 -- 172.21.15.7:0/2508839855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8024032440 con 0x7f80300d3250 2024-09-15T23:37:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.784+0000 7f802e7fc640 1 -- 172.21.15.7:0/2508839855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024032810 con 0x7f80300d3250 2024-09-15T23:37:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.784+0000 7f8035b3b640 1 -- 172.21.15.7:0/2508839855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80300d3250 msgr2=0x7f80300d51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.784+0000 7f8035b3b640 1 --2- 172.21.15.7:0/2508839855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80300d3250 0x7f80300d51a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8024009790 tx=0x7f802402a270 comp rx=0 tx=0).stop 2024-09-15T23:37:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.784+0000 7f8035b3b640 1 -- 172.21.15.7:0/2508839855 shutdown_connections 2024-09-15T23:37:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.784+0000 7f8035b3b640 1 --2- 172.21.15.7:0/2508839855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80300d3250 0x7f80300d51a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.784+0000 7f8035b3b640 1 -- 172.21.15.7:0/2508839855 >> 172.21.15.7:0/2508839855 conn(0x7f803008bfd0 msgr2=0x7f80300d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.784+0000 7f8035b3b640 1 -- 172.21.15.7:0/2508839855 shutdown_connections 2024-09-15T23:37:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.784+0000 7f8035b3b640 1 -- 172.21.15.7:0/2508839855 wait complete. 2024-09-15T23:37:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.785+0000 7f8035b3b640 1 Processor -- start 2024-09-15T23:37:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.785+0000 7f8035b3b640 1 -- start start 2024-09-15T23:37:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f8035b3b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300cfed0 0x7f8030150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f8035b3b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80300d3250 0x7f803009e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f8035b3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8030161e60 0x7f80300a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f8035b3b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80300d1150 con 0x7f8030161e60 2024-09-15T23:37:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f8035b3b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80300d0e70 con 0x7f80300d3250 2024-09-15T23:37:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f8035b3b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f80300d1670 con 0x7f80300cfed0 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f802ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8030161e60 0x7f80300a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f802ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8030161e60 0x7f80300a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53626/0 (socket says 172.21.15.7:53626) 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f802ffff640 1 -- 172.21.15.7:0/1516551839 learned_addr learned my addr 172.21.15.7:0/1516551839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.786+0000 7f802f7fe640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300cfed0 0x7f8030150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f802effd640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80300d3250 0x7f803009e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f802ffff640 1 -- 172.21.15.7:0/1516551839 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300cfed0 msgr2=0x7f8030150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f802ffff640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300cfed0 0x7f8030150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f802ffff640 1 -- 172.21.15.7:0/1516551839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80300d3250 msgr2=0x7f803009e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f802ffff640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80300d3250 0x7f803009e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f802ffff640 1 -- 172.21.15.7:0/1516551839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80300a0bc0 con 0x7f8030161e60 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f802ffff640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8030161e60 0x7f80300a1c60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f802800c7b0 tx=0x7f802800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f8017fff640 1 -- 172.21.15.7:0/1516551839 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028017070 con 0x7f8030161e60 2024-09-15T23:37:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f8017fff640 1 -- 172.21.15.7:0/1516551839 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8028003da0 con 0x7f8030161e60 2024-09-15T23:37:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80300a0ea0 con 0x7f8030161e60 2024-09-15T23:37:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f8017fff640 1 -- 172.21.15.7:0/1516551839 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80280052b0 con 0x7f8030161e60 2024-09-15T23:37:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.787+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80300a1460 con 0x7f8030161e60 2024-09-15T23:37:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.788+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80300d1a40 con 0x7f8030161e60 2024-09-15T23:37:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.789+0000 7f8017fff640 1 -- 172.21.15.7:0/1516551839 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f802801a070 con 0x7f8030161e60 2024-09-15T23:37:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.789+0000 7f8017fff640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f801c06c2b0 0x7f801c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.789+0000 7f802f7fe640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f801c06c2b0 0x7f801c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.789+0000 7f8017fff640 1 -- 172.21.15.7:0/1516551839 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f802808a9b0 con 0x7f8030161e60 2024-09-15T23:37:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.790+0000 7f802f7fe640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f801c06c2b0 0x7f801c06e770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8020000f60 tx=0x7f8020009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.794+0000 7f8017fff640 1 -- 172.21.15.7:0/1516551839 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8028059240 con 0x7f8030161e60 2024-09-15T23:37:12.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.970+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f80300d2560 con 0x7f8030161e60 2024-09-15T23:37:12.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.970+0000 7f8017fff640 1 -- 172.21.15.7:0/1516551839 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f8028058b40 con 0x7f8030161e60 2024-09-15T23:37:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:12.972 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-15T23:37:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-15T23:37:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-15T23:37:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:12.972 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-15T23:37:12.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.972+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f801c06c2b0 msgr2=0x7f801c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.972+0000 7f8035b3b640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f801c06c2b0 0x7f801c06e770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8020000f60 tx=0x7f8020009040 comp rx=0 tx=0).stop 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.972+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8030161e60 msgr2=0x7f80300a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.972+0000 7f8035b3b640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8030161e60 0x7f80300a1c60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f802800c7b0 tx=0x7f802800cc70 comp rx=0 tx=0).stop 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.973+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 shutdown_connections 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.973+0000 7f8035b3b640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8030161e60 0x7f80300a1c60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.973+0000 7f8035b3b640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80300d3250 0x7f803009e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.973+0000 7f8035b3b640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f801c06c2b0 0x7f801c06e770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.973+0000 7f8035b3b640 1 --2- 172.21.15.7:0/1516551839 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300cfed0 0x7f8030150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.973+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 >> 172.21.15.7:0/1516551839 conn(0x7f803008bfd0 msgr2=0x7f803015e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.973+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 shutdown_connections 2024-09-15T23:37:12.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:12.973+0000 7f8035b3b640 1 -- 172.21.15.7:0/1516551839 wait complete. 2024-09-15T23:37:12.985 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-15T23:37:12.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-15T23:37:13.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.091+0000 7fa31af0a640 1 Processor -- start 2024-09-15T23:37:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.091+0000 7fa31af0a640 1 -- start start 2024-09-15T23:37:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.091+0000 7fa31af0a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa314161cb0 0x7fa3140b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.091+0000 7fa31af0a640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa314162090 con 0x7fa3140b1870 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa31af0a640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3140b3f30 con 0x7fa314162a30 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa31af0a640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa3140b4b40 con 0x7fa314161cb0 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa318ca6640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa314161cb0 0x7fa3140b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa318ca6640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa314161cb0 0x7fa3140b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53636/0 (socket says 172.21.15.7:53636) 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa318ca6640 1 -- 172.21.15.7:0/3931912996 learned_addr learned my addr 172.21.15.7:0/3931912996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa313fff640 1 --1- 172.21.15.7:0/3931912996 >> v1:172.21.15.7:6789/0 conn(0x7fa3140b1870 0x7fa314162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49994/0 (socket says 172.21.15.7:49994) 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa318ca6640 1 -- 172.21.15.7:0/3931912996 >> v1:172.21.15.18:6789/0 conn(0x7fa314162a30 legacy=0x7fa3140b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa318ca6640 1 -- 172.21.15.7:0/3931912996 >> v1:172.21.15.7:6789/0 conn(0x7fa3140b1870 legacy=0x7fa314162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa318ca6640 1 -- 172.21.15.7:0/3931912996 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3140b4210 con 0x7fa314161cb0 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.092+0000 7fa3137fe640 1 -- 172.21.15.7:0/3931912996 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645508810 0 0) 0x7fa314162090 con 0x7fa3140b1870 2024-09-15T23:37:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.093+0000 7fa318ca6640 1 --2- 172.21.15.7:0/3931912996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 0x7fa3140b1330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa304009870 tx=0x7fa30402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e7c40eb853b914 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.093+0000 7fa3137fe640 1 -- 172.21.15.7:0/3931912996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304037070 con 0x7fa314161cb0 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.093+0000 7fa3137fe640 1 -- 172.21.15.7:0/3931912996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30402ae50 con 0x7fa314161cb0 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.093+0000 7fa3137fe640 1 -- 172.21.15.7:0/3931912996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304032860 con 0x7fa314161cb0 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.093+0000 7fa31af0a640 1 -- 172.21.15.7:0/3931912996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 msgr2=0x7fa3140b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.093+0000 7fa31af0a640 1 --2- 172.21.15.7:0/3931912996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 0x7fa3140b1330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa304009870 tx=0x7fa30402a300 comp rx=0 tx=0).stop 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.093+0000 7fa31af0a640 1 -- 172.21.15.7:0/3931912996 shutdown_connections 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.093+0000 7fa31af0a640 1 --2- 172.21.15.7:0/3931912996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 0x7fa3140b1330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.093+0000 7fa31af0a640 1 -- 172.21.15.7:0/3931912996 >> 172.21.15.7:0/3931912996 conn(0x7fa31408bfd0 msgr2=0x7fa3140d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.094+0000 7fa31af0a640 1 -- 172.21.15.7:0/3931912996 shutdown_connections 2024-09-15T23:37:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.094+0000 7fa31af0a640 1 -- 172.21.15.7:0/3931912996 wait complete. 2024-09-15T23:37:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.095+0000 7fa31af0a640 1 Processor -- start 2024-09-15T23:37:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.095+0000 7fa31af0a640 1 -- start start 2024-09-15T23:37:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa31af0a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3140b1870 0x7fa3140b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa31af0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 0x7fa3140b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa31af0a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa314162a30 0x7fa314147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa31af0a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3140b3bc0 con 0x7fa314161cb0 2024-09-15T23:37:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa31af0a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa3140b47d0 con 0x7fa3140b1870 2024-09-15T23:37:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa31af0a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa314163000 con 0x7fa314162a30 2024-09-15T23:37:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 0x7fa3140b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 0x7fa3140b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53652/0 (socket says 172.21.15.7:53652) 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa313fff640 1 -- 172.21.15.7:0/2449367914 learned_addr learned my addr 172.21.15.7:0/2449367914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa318ca6640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3140b1870 0x7fa3140b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa3194a7640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa314162a30 0x7fa314147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa313fff640 1 -- 172.21.15.7:0/2449367914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa314162a30 msgr2=0x7fa314147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa313fff640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa314162a30 0x7fa314147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa313fff640 1 -- 172.21.15.7:0/2449367914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3140b1870 msgr2=0x7fa3140b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa313fff640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3140b1870 0x7fa3140b9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa313fff640 1 -- 172.21.15.7:0/2449367914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa314147d10 con 0x7fa314161cb0 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa3194a7640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa314162a30 0x7fa314147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa313fff640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 0x7fa3140b2500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c8c0 tx=0x7fa30800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa3117fa640 1 -- 172.21.15.7:0/2449367914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308018070 con 0x7fa314161cb0 2024-09-15T23:37:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa3117fa640 1 -- 172.21.15.7:0/2449367914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30800a640 con 0x7fa314161cb0 2024-09-15T23:37:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.096+0000 7fa3117fa640 1 -- 172.21.15.7:0/2449367914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30800aa70 con 0x7fa314161cb0 2024-09-15T23:37:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.097+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa314147ff0 con 0x7fa314161cb0 2024-09-15T23:37:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.097+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3140b98b0 con 0x7fa314161cb0 2024-09-15T23:37:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.098+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3141e9840 con 0x7fa314161cb0 2024-09-15T23:37:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.098+0000 7fa3117fa640 1 -- 172.21.15.7:0/2449367914 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa30800ac10 con 0x7fa314161cb0 2024-09-15T23:37:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.099+0000 7fa3117fa640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa30006c2b0 0x7fa30006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.099+0000 7fa3117fa640 1 -- 172.21.15.7:0/2449367914 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fa30808a9a0 con 0x7fa314161cb0 2024-09-15T23:37:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.099+0000 7fa318ca6640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa30006c2b0 0x7fa30006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.099+0000 7fa318ca6640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa30006c2b0 0x7fa30006e770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa304034000 tx=0x7fa30402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.103+0000 7fa3117fa640 1 -- 172.21.15.7:0/2449367914 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa3141e9840 con 0x7fa314161cb0 2024-09-15T23:37:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:13.280+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa3140ceec0 con 0x7fa314161cb0 2024-09-15T23:37:14.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.002+0000 7fa3117fa640 1 -- 172.21.15.7:0/2449367914 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7fa3140ceec0 con 0x7fa314161cb0 2024-09-15T23:37:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-15T23:37:14.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.007+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa30006c2b0 msgr2=0x7fa30006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:14.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.007+0000 7fa31af0a640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa30006c2b0 0x7fa30006e770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa304034000 tx=0x7fa30402e000 comp rx=0 tx=0).stop 2024-09-15T23:37:14.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.007+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 msgr2=0x7fa3140b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:14.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.007+0000 7fa31af0a640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 0x7fa3140b2500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c8c0 tx=0x7fa30800cd80 comp rx=0 tx=0).stop 2024-09-15T23:37:14.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.008+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 shutdown_connections 2024-09-15T23:37:14.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.008+0000 7fa31af0a640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa30006c2b0 0x7fa30006e770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:14.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.008+0000 7fa31af0a640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa314162a30 0x7fa314147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:14.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.008+0000 7fa31af0a640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa314161cb0 0x7fa3140b2500 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:14.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.008+0000 7fa31af0a640 1 --2- 172.21.15.7:0/2449367914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3140b1870 0x7fa3140b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:14.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.008+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 >> 172.21.15.7:0/2449367914 conn(0x7fa31408bfd0 msgr2=0x7fa3140d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.009+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 shutdown_connections 2024-09-15T23:37:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.009+0000 7fa31af0a640 1 -- 172.21.15.7:0/2449367914 wait complete. 2024-09-15T23:37:14.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-15T23:37:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.140+0000 7f008798d640 1 Processor -- start 2024-09-15T23:37:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.140+0000 7f008798d640 1 -- start start 2024-09-15T23:37:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f008798d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0080070ea0 0x7f00800d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f008798d640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f008009bae0 con 0x7f008015e850 2024-09-15T23:37:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f008798d640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00800d07e0 con 0x7f008009b450 2024-09-15T23:37:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f008798d640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f00800d05b0 con 0x7f0080070ea0 2024-09-15T23:37:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f0084f28640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0080070ea0 0x7f00800d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f0084f28640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0080070ea0 0x7f00800d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53662/0 (socket says 172.21.15.7:53662) 2024-09-15T23:37:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f0085729640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f008015e850 0x7f0080070790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50010/0 (socket says 172.21.15.7:50010) 2024-09-15T23:37:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f0084f28640 1 -- 172.21.15.7:0/2110091347 learned_addr learned my addr 172.21.15.7:0/2110091347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f0084f28640 1 -- 172.21.15.7:0/2110091347 >> v1:172.21.15.18:6789/0 conn(0x7f008009b450 legacy=0x7f008006cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f0084f28640 1 -- 172.21.15.7:0/2110091347 >> v1:172.21.15.7:6789/0 conn(0x7f008015e850 legacy=0x7f0080070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f0077fff640 1 -- 172.21.15.7:0/2110091347 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1961715800 0 0) 0x7f008009bae0 con 0x7f008015e850 2024-09-15T23:37:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.141+0000 7f0084f28640 1 -- 172.21.15.7:0/2110091347 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00800d0ac0 con 0x7f0080070ea0 2024-09-15T23:37:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.142+0000 7f0084f28640 1 --2- 172.21.15.7:0/2110091347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0080070ea0 0x7f00800d15a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0070009870 tx=0x7f007002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35735606f85d2d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.142+0000 7f0077fff640 1 -- 172.21.15.7:0/2110091347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070037070 con 0x7f0080070ea0 2024-09-15T23:37:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.142+0000 7f0077fff640 1 -- 172.21.15.7:0/2110091347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f007002ae50 con 0x7f0080070ea0 2024-09-15T23:37:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.142+0000 7f0077fff640 1 -- 172.21.15.7:0/2110091347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070032860 con 0x7f0080070ea0 2024-09-15T23:37:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.143+0000 7f008798d640 1 -- 172.21.15.7:0/2110091347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0080070ea0 msgr2=0x7f00800d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.143+0000 7f008798d640 1 --2- 172.21.15.7:0/2110091347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0080070ea0 0x7f00800d15a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0070009870 tx=0x7f007002a300 comp rx=0 tx=0).stop 2024-09-15T23:37:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.143+0000 7f008798d640 1 -- 172.21.15.7:0/2110091347 shutdown_connections 2024-09-15T23:37:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.143+0000 7f008798d640 1 --2- 172.21.15.7:0/2110091347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0080070ea0 0x7f00800d15a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.143+0000 7f008798d640 1 -- 172.21.15.7:0/2110091347 >> 172.21.15.7:0/2110091347 conn(0x7f00800d2420 msgr2=0x7f00800d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.143+0000 7f008798d640 1 -- 172.21.15.7:0/2110091347 shutdown_connections 2024-09-15T23:37:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.143+0000 7f008798d640 1 -- 172.21.15.7:0/2110091347 wait complete. 2024-09-15T23:37:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.144+0000 7f008798d640 1 Processor -- start 2024-09-15T23:37:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.144+0000 7f008798d640 1 -- start start 2024-09-15T23:37:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f008798d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0080070ea0 0x7f008015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f008798d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008009b450 0x7f008014f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f008798d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008015e850 0x7f008014fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f008798d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f008015ec30 con 0x7f008009b450 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f008798d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f00800d1110 con 0x7f008015e850 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f008798d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f00800b18c0 con 0x7f0080070ea0 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0084f28640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008009b450 0x7f008014f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0084f28640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008009b450 0x7f008014f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53674/0 (socket says 172.21.15.7:53674) 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0084f28640 1 -- 172.21.15.7:0/1764478661 learned_addr learned my addr 172.21.15.7:0/1764478661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0085729640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0080070ea0 0x7f008015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0085f2a640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008015e850 0x7f008014fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0084f28640 1 -- 172.21.15.7:0/1764478661 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0080070ea0 msgr2=0x7f008015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0084f28640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0080070ea0 0x7f008015d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0084f28640 1 -- 172.21.15.7:0/1764478661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008015e850 msgr2=0x7f008014fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0084f28640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008015e850 0x7f008014fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.145+0000 7f0084f28640 1 -- 172.21.15.7:0/1764478661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0080161e80 con 0x7f008009b450 2024-09-15T23:37:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.146+0000 7f0084f28640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008009b450 0x7f008014f660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00700090b0 tx=0x7f00700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.146+0000 7f0075ffb640 1 -- 172.21.15.7:0/1764478661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070037040 con 0x7f008009b450 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.146+0000 7f0075ffb640 1 -- 172.21.15.7:0/1764478661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0070004230 con 0x7f008009b450 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.146+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0080162160 con 0x7f008009b450 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.146+0000 7f0075ffb640 1 -- 172.21.15.7:0/1764478661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070032e60 con 0x7f008009b450 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.146+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0080162720 con 0x7f008009b450 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.147+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00800b1c50 con 0x7f008009b450 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.147+0000 7f0075ffb640 1 -- 172.21.15.7:0/1764478661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f007003a040 con 0x7f008009b450 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.148+0000 7f0075ffb640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006406c2b0 0x7f006406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.148+0000 7f0075ffb640 1 -- 172.21.15.7:0/1764478661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7f00700abac0 con 0x7f008009b450 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.148+0000 7f0085729640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006406c2b0 0x7f006406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.148+0000 7f0085729640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006406c2b0 0x7f006406e770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f007c000f60 tx=0x7f007c007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.153+0000 7f0075ffb640 1 -- 172.21.15.7:0/1764478661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0070042020 con 0x7f008009b450 2024-09-15T23:37:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:14.331+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f00800b83a0 con 0x7f008009b450 2024-09-15T23:37:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.005+0000 7f0075ffb640 1 -- 172.21.15.7:0/1764478661 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7f0070079ba0 con 0x7f008009b450 2024-09-15T23:37:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-15T23:37:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.008+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006406c2b0 msgr2=0x7f006406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.008+0000 7f008798d640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006406c2b0 0x7f006406e770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f007c000f60 tx=0x7f007c007e40 comp rx=0 tx=0).stop 2024-09-15T23:37:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.008+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008009b450 msgr2=0x7f008014f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.008+0000 7f008798d640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008009b450 0x7f008014f660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00700090b0 tx=0x7f00700046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.010+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 shutdown_connections 2024-09-15T23:37:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.010+0000 7f008798d640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008015e850 0x7f008014fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.010+0000 7f008798d640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f008009b450 0x7f008014f660 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00700090b0 tx=0x7f00700046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.010+0000 7f008798d640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f006406c2b0 0x7f006406e770 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.010+0000 7f008798d640 1 --2- 172.21.15.7:0/1764478661 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0080070ea0 0x7f008015d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.010+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 >> 172.21.15.7:0/1764478661 conn(0x7f00800d2420 msgr2=0x7f00800b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.011+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 shutdown_connections 2024-09-15T23:37:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.012+0000 7f008798d640 1 -- 172.21.15.7:0/1764478661 wait complete. 2024-09-15T23:37:15.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-15T23:37:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.149+0000 7f22db986640 1 Processor -- start 2024-09-15T23:37:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.149+0000 7f22db986640 1 -- start start 2024-09-15T23:37:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22db986640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f22d41558e0 0x7f22d40ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22db986640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d415f600 0x7f22d4072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22db986640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22d40af270 con 0x7f22d406e060 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22db986640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f22d40b3160 con 0x7f22d41558e0 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22db986640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f22d40af550 con 0x7f22d415f600 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22d8f21640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f22d406e060 0x7f22d415eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50014/0 (socket says 172.21.15.7:50014) 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22d8f21640 1 -- 172.21.15.7:0/3106240299 learned_addr learned my addr 172.21.15.7:0/3106240299 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22d9722640 1 --2- 172.21.15.7:0/3106240299 >> v2:172.21.15.18:3300/0 conn(0x7f22d41558e0 0x7f22d40ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22d9f23640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d415f600 0x7f22d4072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1011502330 0 0) 0x7f22d40af270 con 0x7f22d406e060 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.150+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22c0005730 con 0x7f22d406e060 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1880083704 0 0) 0x7f22c0005730 con 0x7f22d406e060 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d415f600 msgr2=0x7f22d4072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d415f600 0x7f22d4072b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 >> v2:172.21.15.18:3300/0 conn(0x7f22d41558e0 msgr2=0x7f22d40ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 --2- 172.21.15.7:0/3106240299 >> v2:172.21.15.18:3300/0 conn(0x7f22d41558e0 0x7f22d40ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d4072020 con 0x7f22d406e060 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f22c4001670 con 0x7f22d406e060 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 >> v1:172.21.15.7:6789/0 conn(0x7f22d406e060 legacy=0x7f22d415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22c0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d415f600 0x7f22c0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22c0006f50 0x7f22c00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22d40af550 con 0x7f22c0005bc0 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f22c000dc00 con 0x7f22c0006f50 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f22c000de00 con 0x7f22d415f600 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22d9722640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22c0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22d8f21640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d415f600 0x7f22c0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22d9f23640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22c0006f50 0x7f22c00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22d9722640 1 -- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d415f600 msgr2=0x7f22c0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22d9722640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d415f600 0x7f22c0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22d9722640 1 -- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22c0006f50 msgr2=0x7f22c00073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22d9722640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22c0006f50 0x7f22c00073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22d9722640 1 -- 172.21.15.7:0/3106240299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c0005730 con 0x7f22c0005bc0 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.151+0000 7f22d9f23640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22c0006f50 0x7f22c00073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.152+0000 7f22d9722640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22c0005fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009ac0 tx=0x7f22c0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e91966e96121329 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.152+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0036070 con 0x7f22c0005bc0 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.152+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d0031400 con 0x7f22c0005bc0 2024-09-15T23:37:15.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.152+0000 7f22cbfff640 1 -- 172.21.15.7:0/3106240299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0009690 con 0x7f22c0005bc0 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.152+0000 7f22db986640 1 -- 172.21.15.7:0/3106240299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 msgr2=0x7f22c0005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.152+0000 7f22db986640 1 --2- 172.21.15.7:0/3106240299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22c0005fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009ac0 tx=0x7f22c0007a40 comp rx=0 tx=0).stop 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.152+0000 7f22d9f23640 1 -- 172.21.15.7:0/3106240299 reap_dead start 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.153+0000 7f22db986640 1 -- 172.21.15.7:0/3106240299 shutdown_connections 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.153+0000 7f22db986640 1 -- 172.21.15.7:0/3106240299 >> 172.21.15.7:0/3106240299 conn(0x7f22d40b1e60 msgr2=0x7f22d40b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.153+0000 7f22db986640 1 -- 172.21.15.7:0/3106240299 shutdown_connections 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.153+0000 7f22db986640 1 -- 172.21.15.7:0/3106240299 wait complete. 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.154+0000 7f22db986640 1 Processor -- start 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.154+0000 7f22db986640 1 -- start start 2024-09-15T23:37:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22db986640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22d40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22db986640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22d40af700 0x7f22d4150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22db986640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d40cef90 0x7f22d415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22db986640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22d4155cc0 con 0x7f22c0005bc0 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22db986640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f22d40aeef0 con 0x7f22d40af700 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22db986640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f22d4155e40 con 0x7f22d40cef90 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9722640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22d40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9722640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22d40b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53686/0 (socket says 172.21.15.7:53686) 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9722640 1 -- 172.21.15.7:0/1447995361 learned_addr learned my addr 172.21.15.7:0/1447995361 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9f23640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d40cef90 0x7f22d415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d8f21640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22d40af700 0x7f22d4150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9722640 1 -- 172.21.15.7:0/1447995361 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d40cef90 msgr2=0x7f22d415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9722640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d40cef90 0x7f22d415bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9722640 1 -- 172.21.15.7:0/1447995361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22d40af700 msgr2=0x7f22d4150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9722640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22d40af700 0x7f22d4150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9722640 1 -- 172.21.15.7:0/1447995361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d4158590 con 0x7f22c0005bc0 2024-09-15T23:37:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22d9722640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22d40b4d00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f22d0034040 tx=0x7f22d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22c9ffb640 1 -- 172.21.15.7:0/1447995361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0040020 con 0x7f22c0005bc0 2024-09-15T23:37:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22c9ffb640 1 -- 172.21.15.7:0/1447995361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d0003950 con 0x7f22c0005bc0 2024-09-15T23:37:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d4158870 con 0x7f22c0005bc0 2024-09-15T23:37:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.155+0000 7f22c9ffb640 1 -- 172.21.15.7:0/1447995361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0036040 con 0x7f22c0005bc0 2024-09-15T23:37:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.156+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d4158e30 con 0x7f22c0005bc0 2024-09-15T23:37:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.157+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d40ae9b0 con 0x7f22c0005bc0 2024-09-15T23:37:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.157+0000 7f22c9ffb640 1 -- 172.21.15.7:0/1447995361 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f22d0043020 con 0x7f22c0005bc0 2024-09-15T23:37:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.158+0000 7f22c9ffb640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f22b806c2e0 0x7f22b806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.158+0000 7f22c9ffb640 1 -- 172.21.15.7:0/1447995361 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f22d00ab880 con 0x7f22c0005bc0 2024-09-15T23:37:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.158+0000 7f22d8f21640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f22b806c2e0 0x7f22b806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.158+0000 7f22d8f21640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f22b806c2e0 0x7f22b806e7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f22d41f8cc0 tx=0x7f22c4007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.162+0000 7f22c9ffb640 1 -- 172.21.15.7:0/1447995361 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f22d004a030 con 0x7f22c0005bc0 2024-09-15T23:37:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:15.373+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f22d40cff30 con 0x7f22c0005bc0 2024-09-15T23:37:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.024+0000 7f22c9ffb640 1 -- 172.21.15.7:0/1447995361 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7f22d0079870 con 0x7f22c0005bc0 2024-09-15T23:37:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 5 and data pool 6 2024-09-15T23:37:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f22b806c2e0 msgr2=0x7f22b806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f22b806c2e0 0x7f22b806e7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f22d41f8cc0 tx=0x7f22c4007040 comp rx=0 tx=0).stop 2024-09-15T23:37:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 msgr2=0x7f22d40b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22d40b4d00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f22d0034040 tx=0x7f22d0004730 comp rx=0 tx=0).stop 2024-09-15T23:37:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 shutdown_connections 2024-09-15T23:37:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f22b806c2e0 0x7f22b806e7a0 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f22d41f8cc0 tx=0x7f22c4007040 comp rx=0 tx=0).stop 2024-09-15T23:37:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22d40cef90 0x7f22d415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22d40af700 0x7f22d4150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 --2- 172.21.15.7:0/1447995361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c0005bc0 0x7f22d40b4d00 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f22d0034040 tx=0x7f22d0004730 comp rx=0 tx=0).stop 2024-09-15T23:37:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.034+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 >> 172.21.15.7:0/1447995361 conn(0x7f22d40b1e60 msgr2=0x7f22d415ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.042+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 shutdown_connections 2024-09-15T23:37:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.042+0000 7f22db986640 1 -- 172.21.15.7:0/1447995361 wait complete. 2024-09-15T23:37:16.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:37:16.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.170+0000 7fc8be653640 1 Processor -- start 2024-09-15T23:37:16.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.170+0000 7fc8be653640 1 -- start start 2024-09-15T23:37:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8be653640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc8b80b31c0 0x7fc8b80d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8be653640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc8b815ca20 0x7fc8b80d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8be653640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b806dfa0 con 0x7fc8b8158e60 2024-09-15T23:37:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8be653640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b8159240 con 0x7fc8b815ca20 2024-09-15T23:37:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8be653640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b806e210 con 0x7fc8b80b31c0 2024-09-15T23:37:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8bcbf0640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc8b815ca20 0x7fc8b80d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8bcbf0640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc8b815ca20 0x7fc8b80d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53698/0 (socket says 172.21.15.7:53698) 2024-09-15T23:37:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8bcbf0640 1 -- 172.21.15.7:0/840021389 learned_addr learned my addr 172.21.15.7:0/840021389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8affff640 1 --2- 172.21.15.7:0/840021389 >> v2:172.21.15.18:3300/0 conn(0x7fc8b80b31c0 0x7fc8b80d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8bcbf0640 1 -- 172.21.15.7:0/840021389 >> v2:172.21.15.18:3300/0 conn(0x7fc8b80b31c0 msgr2=0x7fc8b80d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8bcbf0640 1 --2- 172.21.15.7:0/840021389 >> v2:172.21.15.18:3300/0 conn(0x7fc8b80b31c0 0x7fc8b80d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8bcbf0640 1 -- 172.21.15.7:0/840021389 >> v1:172.21.15.18:6789/0 conn(0x7fc8b8158e60 legacy=0x7fc8b80b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.171+0000 7fc8bcbf0640 1 -- 172.21.15.7:0/840021389 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b806e4f0 con 0x7fc8b815ca20 2024-09-15T23:37:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.172+0000 7fc8bcbf0640 1 --2- 172.21.15.7:0/840021389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b815ca20 0x7fc8b80d0b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8b815b860 tx=0x7fc8b002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2267e3d11f2840 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.172+0000 7fc8b77fe640 1 -- 172.21.15.7:0/840021389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0036070 con 0x7fc8b815ca20 2024-09-15T23:37:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.172+0000 7fc8b77fe640 1 -- 172.21.15.7:0/840021389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b0031600 con 0x7fc8b815ca20 2024-09-15T23:37:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.172+0000 7fc8b77fe640 1 -- 172.21.15.7:0/840021389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00319d0 con 0x7fc8b815ca20 2024-09-15T23:37:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.173+0000 7fc8be653640 1 -- 172.21.15.7:0/840021389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b815ca20 msgr2=0x7fc8b80d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.173+0000 7fc8be653640 1 --2- 172.21.15.7:0/840021389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b815ca20 0x7fc8b80d0b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8b815b860 tx=0x7fc8b002a780 comp rx=0 tx=0).stop 2024-09-15T23:37:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.173+0000 7fc8be653640 1 -- 172.21.15.7:0/840021389 shutdown_connections 2024-09-15T23:37:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.173+0000 7fc8be653640 1 --2- 172.21.15.7:0/840021389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b815ca20 0x7fc8b80d0b30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.173+0000 7fc8be653640 1 --2- 172.21.15.7:0/840021389 >> v2:172.21.15.18:3300/0 conn(0x7fc8b80b31c0 0x7fc8b80d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.173+0000 7fc8be653640 1 -- 172.21.15.7:0/840021389 >> 172.21.15.7:0/840021389 conn(0x7fc8b80b21e0 msgr2=0x7fc8b80b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.173+0000 7fc8be653640 1 -- 172.21.15.7:0/840021389 shutdown_connections 2024-09-15T23:37:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.173+0000 7fc8be653640 1 -- 172.21.15.7:0/840021389 wait complete. 2024-09-15T23:37:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.174+0000 7fc8be653640 1 Processor -- start 2024-09-15T23:37:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.174+0000 7fc8be653640 1 -- start start 2024-09-15T23:37:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8be653640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b80b31c0 0x7fc8b81568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8be653640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b8158e60 0x7fc8b809a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8be653640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc8b815ca20 0x7fc8b809a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8be653640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b80b3d90 con 0x7fc8b8158e60 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8be653640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b806eab0 con 0x7fc8b80b31c0 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8be653640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b80b3f10 con 0x7fc8b815ca20 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b8158e60 0x7fc8b809a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b8158e60 0x7fc8b809a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53706/0 (socket says 172.21.15.7:53706) 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8affff640 1 -- 172.21.15.7:0/3811469299 learned_addr learned my addr 172.21.15.7:0/3811469299 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8b7fff640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b80b31c0 0x7fc8b81568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8bcbf0640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc8b815ca20 0x7fc8b809a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.175+0000 7fc8affff640 1 -- 172.21.15.7:0/3811469299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc8b815ca20 msgr2=0x7fc8b809a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8affff640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc8b815ca20 0x7fc8b809a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8affff640 1 -- 172.21.15.7:0/3811469299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b80b31c0 msgr2=0x7fc8b81568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8affff640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b80b31c0 0x7fc8b81568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8affff640 1 -- 172.21.15.7:0/3811469299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b81f4970 con 0x7fc8b8158e60 2024-09-15T23:37:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8affff640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b8158e60 0x7fc8b809a370 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000d800 tx=0x7fc8a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8b57fa640 1 -- 172.21.15.7:0/3811469299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000b6c0 con 0x7fc8b8158e60 2024-09-15T23:37:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8b57fa640 1 -- 172.21.15.7:0/3811469299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a0019070 con 0x7fc8b8158e60 2024-09-15T23:37:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b81f4c50 con 0x7fc8b8158e60 2024-09-15T23:37:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8b57fa640 1 -- 172.21.15.7:0/3811469299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000ba80 con 0x7fc8b8158e60 2024-09-15T23:37:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.176+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b81f5210 con 0x7fc8b8158e60 2024-09-15T23:37:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.177+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b80b3680 con 0x7fc8b8158e60 2024-09-15T23:37:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.178+0000 7fc8b57fa640 1 -- 172.21.15.7:0/3811469299 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc8a000bc20 con 0x7fc8b8158e60 2024-09-15T23:37:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.178+0000 7fc8b57fa640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc89806c2b0 0x7fc89806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.178+0000 7fc8b57fa640 1 -- 172.21.15.7:0/3811469299 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc8a008b930 con 0x7fc8b8158e60 2024-09-15T23:37:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.178+0000 7fc8b7fff640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc89806c2b0 0x7fc89806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.178+0000 7fc8b7fff640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc89806c2b0 0x7fc89806e770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80b1120 tx=0x7fc8a8007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.182+0000 7fc8b57fa640 1 -- 172.21.15.7:0/3811469299 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc8a00598c0 con 0x7fc8b8158e60 2024-09-15T23:37:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.357+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8b80d0590 con 0x7fc8b8158e60 2024-09-15T23:37:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.358+0000 7fc8b57fa640 1 -- 172.21.15.7:0/3811469299 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16919 (secure 0 0 0) 0x7fc8a005dbc0 con 0x7fc8b8158e60 2024-09-15T23:37:16.359 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:16.360 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:37:15.374132+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:37:13.310698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-15T23:37:14.361918+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:37:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc89806c2b0 msgr2=0x7fc89806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc89806c2b0 0x7fc89806e770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80b1120 tx=0x7fc8a8007e30 comp rx=0 tx=0).stop 2024-09-15T23:37:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b8158e60 msgr2=0x7fc8b809a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b8158e60 0x7fc8b809a370 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000d800 tx=0x7fc8a000dcc0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 shutdown_connections 2024-09-15T23:37:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc89806c2b0 0x7fc89806e770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc8b815ca20 0x7fc8b809a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b8158e60 0x7fc8b809a370 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 --2- 172.21.15.7:0/3811469299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b80b31c0 0x7fc8b81568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 >> 172.21.15.7:0/3811469299 conn(0x7fc8b80b21e0 msgr2=0x7fc8b815b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 shutdown_connections 2024-09-15T23:37:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.361+0000 7fc8be653640 1 -- 172.21.15.7:0/3811469299 wait complete. 2024-09-15T23:37:16.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:37:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.474+0000 7f2a4f3a6640 1 Processor -- start 2024-09-15T23:37:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.474+0000 7f2a4f3a6640 1 -- start start 2024-09-15T23:37:16.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.475+0000 7f2a4f3a6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a480d32a0 0x7f2a48169280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.475+0000 7f2a4f3a6640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a480b3d10 con 0x7f2a4815e6d0 2024-09-15T23:37:16.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.475+0000 7f2a4f3a6640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a480cfcd0 con 0x7f2a480d51c0 2024-09-15T23:37:16.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.475+0000 7f2a4f3a6640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f2a480d28c0 con 0x7f2a480d32a0 2024-09-15T23:37:16.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.475+0000 7f2a4d142640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2a4815e6d0 0x7f2a480d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50016/0 (socket says 172.21.15.7:50016) 2024-09-15T23:37:16.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.475+0000 7f2a4d142640 1 -- 172.21.15.7:0/1341904442 learned_addr learned my addr 172.21.15.7:0/1341904442 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:16.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.475+0000 7f2a4d943640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a480d32a0 0x7f2a48169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.475+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641309192 0 0) 0x7f2a480b3d10 con 0x7f2a4815e6d0 2024-09-15T23:37:16.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a34003520 con 0x7f2a4815e6d0 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1496244955 0 0) 0x7f2a38001160 con 0x7f2a480d51c0 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a480b3d10 con 0x7f2a480d51c0 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2883456189 0 0) 0x7f2a34003520 con 0x7f2a4815e6d0 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a480d32a0 msgr2=0x7f2a48169280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a480d32a0 0x7f2a48169280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 >> v1:172.21.15.18:6789/0 conn(0x7f2a480d51c0 legacy=0x7f2a480d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a480cff50 con 0x7f2a4815e6d0 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f2a44001670 con 0x7f2a4815e6d0 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 >> v1:172.21.15.7:6789/0 conn(0x7f2a4815e6d0 legacy=0x7f2a480d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a480d32a0 0x7f2a34003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a34004120 0x7f2a34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a34004d40 0x7f2a340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a480d28c0 con 0x7f2a34004120 2024-09-15T23:37:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3400bb00 con 0x7f2a34004d40 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3400bd00 con 0x7f2a480d32a0 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a4c941640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a34004120 0x7f2a34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a4d142640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a480d32a0 0x7f2a34003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a4d943640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a34004d40 0x7f2a340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a4c941640 1 -- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a480d32a0 msgr2=0x7f2a34003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a4c941640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a480d32a0 0x7f2a34003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a4c941640 1 -- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a34004d40 msgr2=0x7f2a340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a4c941640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a34004d40 0x7f2a340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.476+0000 7f2a4c941640 1 -- 172.21.15.7:0/1341904442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a34003520 con 0x7f2a34004120 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.477+0000 7f2a4c941640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a34004120 0x7f2a34004800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a480d55d0 tx=0x7f2a34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ff124be41a3e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.477+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38001710 con 0x7f2a34004120 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.477+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3802ad30 con 0x7f2a34004120 2024-09-15T23:37:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.477+0000 7f2a3ffff640 1 -- 172.21.15.7:0/1341904442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380052f0 con 0x7f2a34004120 2024-09-15T23:37:16.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.478+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/1341904442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a34004120 msgr2=0x7f2a34004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.478+0000 7f2a4f3a6640 1 --2- 172.21.15.7:0/1341904442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a34004120 0x7f2a34004800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a480d55d0 tx=0x7f2a34005830 comp rx=0 tx=0).stop 2024-09-15T23:37:16.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.478+0000 7f2a4d943640 1 -- 172.21.15.7:0/1341904442 reap_dead start 2024-09-15T23:37:16.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.478+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/1341904442 shutdown_connections 2024-09-15T23:37:16.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.478+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/1341904442 >> 172.21.15.7:0/1341904442 conn(0x7f2a4808bfd0 msgr2=0x7f2a480d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:16.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.478+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/1341904442 shutdown_connections 2024-09-15T23:37:16.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.478+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/1341904442 wait complete. 2024-09-15T23:37:16.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.479+0000 7f2a4f3a6640 1 Processor -- start 2024-09-15T23:37:16.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.479+0000 7f2a4f3a6640 1 -- start start 2024-09-15T23:37:16.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4f3a6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a34004120 0x7f2a480d0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4f3a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a48150c60 0x7f2a480d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4f3a6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a480cb6b0 0x7f2a480cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4f3a6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a480d0510 con 0x7f2a48150c60 2024-09-15T23:37:16.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4f3a6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4815eab0 con 0x7f2a480cb6b0 2024-09-15T23:37:16.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4f3a6640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f2a480b51d0 con 0x7f2a34004120 2024-09-15T23:37:16.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4c941640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a48150c60 0x7f2a480d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4c941640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a48150c60 0x7f2a480d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53724/0 (socket says 172.21.15.7:53724) 2024-09-15T23:37:16.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4c941640 1 -- 172.21.15.7:0/2068282353 learned_addr learned my addr 172.21.15.7:0/2068282353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4d142640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a34004120 0x7f2a480d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4d943640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a480cb6b0 0x7f2a480cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4c941640 1 -- 172.21.15.7:0/2068282353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a34004120 msgr2=0x7f2a480d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4c941640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a34004120 0x7f2a480d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4c941640 1 -- 172.21.15.7:0/2068282353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a480cb6b0 msgr2=0x7f2a480cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4c941640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a480cb6b0 0x7f2a480cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4c941640 1 -- 172.21.15.7:0/2068282353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a480b38a0 con 0x7f2a48150c60 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.480+0000 7f2a4c941640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a48150c60 0x7f2a480d0bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2a38000c00 tx=0x7f2a380014f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.481+0000 7f2a3dffb640 1 -- 172.21.15.7:0/2068282353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3803f030 con 0x7f2a48150c60 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.481+0000 7f2a3dffb640 1 -- 172.21.15.7:0/2068282353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a380016f0 con 0x7f2a48150c60 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.481+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a480cc290 con 0x7f2a48150c60 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.481+0000 7f2a3dffb640 1 -- 172.21.15.7:0/2068282353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38006e60 con 0x7f2a48150c60 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.481+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a480cc850 con 0x7f2a48150c60 2024-09-15T23:37:16.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.482+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a480d2b90 con 0x7f2a48150c60 2024-09-15T23:37:16.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.483+0000 7f2a3dffb640 1 -- 172.21.15.7:0/2068282353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2a38042070 con 0x7f2a48150c60 2024-09-15T23:37:16.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.483+0000 7f2a3dffb640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2a2c06c2e0 0x7f2a2c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.483+0000 7f2a3dffb640 1 -- 172.21.15.7:0/2068282353 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f2a380aa980 con 0x7f2a48150c60 2024-09-15T23:37:16.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.483+0000 7f2a4d142640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2a2c06c2e0 0x7f2a2c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.484+0000 7f2a4d142640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2a2c06c2e0 0x7f2a2c06e7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2a440017e0 tx=0x7f2a440015b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:16.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.488+0000 7f2a3dffb640 1 -- 172.21.15.7:0/2068282353 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2a3804a020 con 0x7f2a48150c60 2024-09-15T23:37:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.664+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a4806d420 con 0x7f2a48150c60 2024-09-15T23:37:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.665+0000 7f2a3dffb640 1 -- 172.21.15.7:0/2068282353 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16919 (secure 0 0 0) 0x7f2a480cfcd0 con 0x7f2a48150c60 2024-09-15T23:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:37:15.374132+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:37:13.310698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-15T23:37:14.361918+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2a2c06c2e0 msgr2=0x7f2a2c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2a2c06c2e0 0x7f2a2c06e7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2a440017e0 tx=0x7f2a440015b0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a48150c60 msgr2=0x7f2a480d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a48150c60 0x7f2a480d0bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2a38000c00 tx=0x7f2a380014f0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 shutdown_connections 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a480cb6b0 0x7f2a480cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a48150c60 0x7f2a480d0bd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f2a2c06c2e0 0x7f2a2c06e7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 --2- 172.21.15.7:0/2068282353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2a34004120 0x7f2a480d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 >> 172.21.15.7:0/2068282353 conn(0x7f2a4808bfd0 msgr2=0x7f2a480b9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 shutdown_connections 2024-09-15T23:37:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.668+0000 7f2a4f3a6640 1 -- 172.21.15.7:0/2068282353 wait complete. 2024-09-15T23:37:16.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-15T23:37:16.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.784+0000 7fc74b14b640 1 Processor -- start 2024-09-15T23:37:16.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.784+0000 7fc74b14b640 1 -- start start 2024-09-15T23:37:16.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.785+0000 7fc74b14b640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc744073b10 0x7fc744166280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.785+0000 7fc74b14b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7440af270 con 0x7fc7441558e0 2024-09-15T23:37:16.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.785+0000 7fc74b14b640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc744072020 con 0x7fc74406e060 2024-09-15T23:37:16.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.785+0000 7fc74b14b640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fc7440b3160 con 0x7fc744073b10 2024-09-15T23:37:16.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.785+0000 7fc748ee7640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc7441558e0 0x7fc7440ae9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50030/0 (socket says 172.21.15.7:50030) 2024-09-15T23:37:16.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.785+0000 7fc748ee7640 1 -- 172.21.15.7:0/3552769581 learned_addr learned my addr 172.21.15.7:0/3552769581 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914696971 0 0) 0x7fc7440af270 con 0x7fc7441558e0 2024-09-15T23:37:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7496e8640 1 --2- 172.21.15.7:0/3552769581 >> v2:172.21.15.18:3300/0 conn(0x7fc744073b10 0x7fc744166280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc72c005730 con 0x7fc7441558e0 2024-09-15T23:37:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3377463017 0 0) 0x7fc72c005730 con 0x7fc7441558e0 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 >> v2:172.21.15.18:3300/0 conn(0x7fc744073b10 msgr2=0x7fc744166280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 --2- 172.21.15.7:0/3552769581 >> v2:172.21.15.18:3300/0 conn(0x7fc744073b10 0x7fc744166280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 >> v1:172.21.15.18:6789/0 conn(0x7fc74406e060 legacy=0x7fc74415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc744072300 con 0x7fc7441558e0 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7fc734001670 con 0x7fc7441558e0 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 >> v1:172.21.15.7:6789/0 conn(0x7fc7441558e0 legacy=0x7fc7440ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 0x7fc72c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc72c006a10 0x7fc72c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc72c0074f0 0x7fc72c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c00e390 con 0x7fc72c006a10 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c00e540 con 0x7fc72c005bc0 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fc72c00e740 con 0x7fc72c0074f0 2024-09-15T23:37:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc743fff640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc72c006a10 0x7fc72c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc748ee7640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 0x7fc72c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.786+0000 7fc7496e8640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc72c0074f0 0x7fc72c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.787+0000 7fc748ee7640 1 -- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc72c0074f0 msgr2=0x7fc72c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.787+0000 7fc748ee7640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc72c0074f0 0x7fc72c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.787+0000 7fc748ee7640 1 -- 172.21.15.7:0/3552769581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc72c006a10 msgr2=0x7fc72c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.787+0000 7fc748ee7640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc72c006a10 0x7fc72c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.787+0000 7fc748ee7640 1 -- 172.21.15.7:0/3552769581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c005730 con 0x7fc72c005bc0 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.787+0000 7fc743fff640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc72c006a10 0x7fc72c006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.787+0000 7fc7496e8640 1 -- 172.21.15.7:0/3552769581 reap_dead start 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.787+0000 7fc748ee7640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 0x7fc72c005fe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c210 tx=0x7fc72c007a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae186be0d24611cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734007cf0 con 0x7fc72c005bc0 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc734005af0 con 0x7fc72c005bc0 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc7437fe640 1 -- 172.21.15.7:0/3552769581 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734007070 con 0x7fc72c005bc0 2024-09-15T23:37:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc74b14b640 1 -- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 msgr2=0x7fc72c005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc74b14b640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 0x7fc72c005fe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c210 tx=0x7fc72c007a40 comp rx=0 tx=0).stop 2024-09-15T23:37:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc74b14b640 1 -- 172.21.15.7:0/3552769581 shutdown_connections 2024-09-15T23:37:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc74b14b640 1 --2- 172.21.15.7:0/3552769581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 0x7fc72c005fe0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc74b14b640 1 -- 172.21.15.7:0/3552769581 >> 172.21.15.7:0/3552769581 conn(0x7fc7440b1e60 msgr2=0x7fc7440b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc74b14b640 1 -- 172.21.15.7:0/3552769581 shutdown_connections 2024-09-15T23:37:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.788+0000 7fc74b14b640 1 -- 172.21.15.7:0/3552769581 wait complete. 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.789+0000 7fc74b14b640 1 Processor -- start 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.789+0000 7fc74b14b640 1 -- start start 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.790+0000 7fc74b14b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 0x7fc744152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.790+0000 7fc74b14b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7440b1550 0x7fc7441f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.790+0000 7fc74b14b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc744156880 0x7fc744158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.790+0000 7fc74b14b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7440ae9b0 con 0x7fc744156880 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.790+0000 7fc74b14b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc7440af550 con 0x7fc72c005bc0 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.790+0000 7fc74b14b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fc7440aeb30 con 0x7fc7440b1550 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.790+0000 7fc7496e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc744156880 0x7fc744158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.790+0000 7fc7496e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc744156880 0x7fc744158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53746/0 (socket says 172.21.15.7:53746) 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.790+0000 7fc7496e8640 1 -- 172.21.15.7:0/1962363935 learned_addr learned my addr 172.21.15.7:0/1962363935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc743fff640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7440b1550 0x7fc7441f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc748ee7640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 0x7fc744152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc7496e8640 1 -- 172.21.15.7:0/1962363935 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7440b1550 msgr2=0x7fc7441f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc7496e8640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7440b1550 0x7fc7441f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc7496e8640 1 -- 172.21.15.7:0/1962363935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 msgr2=0x7fc744152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc7496e8640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 0x7fc744152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc7496e8640 1 -- 172.21.15.7:0/1962363935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7441571e0 con 0x7fc744156880 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc7496e8640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc744156880 0x7fc744158270 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc74406d9b0 tx=0x7fc73c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc7417fa640 1 -- 172.21.15.7:0/1962363935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c00fc80 con 0x7fc744156880 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc7417fa640 1 -- 172.21.15.7:0/1962363935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc73c00fe20 con 0x7fc744156880 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7440af800 con 0x7fc744156880 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc7417fa640 1 -- 172.21.15.7:0/1962363935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c015670 con 0x7fc744156880 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.791+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7440afdc0 con 0x7fc744156880 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.792+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7440aeec0 con 0x7fc744156880 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.793+0000 7fc7417fa640 1 -- 172.21.15.7:0/1962363935 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc73c015810 con 0x7fc744156880 2024-09-15T23:37:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.793+0000 7fc7417fa640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc73006c2e0 0x7fc73006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.793+0000 7fc7417fa640 1 -- 172.21.15.7:0/1962363935 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc73c08c230 con 0x7fc744156880 2024-09-15T23:37:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.793+0000 7fc748ee7640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc73006c2e0 0x7fc73006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.794+0000 7fc748ee7640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc73006c2e0 0x7fc73006e7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc734032000 tx=0x7fc734009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:16.798+0000 7fc7417fa640 1 -- 172.21.15.7:0/1962363935 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc73c019030 con 0x7fc744156880 2024-09-15T23:37:17.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.010+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc7440b2d10 con 0x7fc744156880 2024-09-15T23:37:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.024+0000 7fc7417fa640 1 -- 172.21.15.7:0/1962363935 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7fc73c010040 con 0x7fc744156880 2024-09-15T23:37:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc73006c2e0 msgr2=0x7fc73006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc73006c2e0 0x7fc73006e7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc734032000 tx=0x7fc734009030 comp rx=0 tx=0).stop 2024-09-15T23:37:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc744156880 msgr2=0x7fc744158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc744156880 0x7fc744158270 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc74406d9b0 tx=0x7fc73c00f1b0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 shutdown_connections 2024-09-15T23:37:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc744156880 0x7fc744158270 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fc73006c2e0 0x7fc73006e7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7440b1550 0x7fc7441f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 --2- 172.21.15.7:0/1962363935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc72c005bc0 0x7fc744152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 >> 172.21.15.7:0/1962363935 conn(0x7fc7440b1e60 msgr2=0x7fc74415adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 shutdown_connections 2024-09-15T23:37:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.026+0000 7fc74b14b640 1 -- 172.21.15.7:0/1962363935 wait complete. 2024-09-15T23:37:17.039 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-15T23:37:17.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-15T23:37:17.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.145+0000 7f9167133640 1 Processor -- start 2024-09-15T23:37:17.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.145+0000 7f9167133640 1 -- start start 2024-09-15T23:37:17.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.145+0000 7f9167133640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f916015e760 0x7f91600d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:17.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.145+0000 7f9167133640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f91600d0540 0x7f91600d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.145+0000 7f9167133640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91600d1440 con 0x7f91600d1030 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.145+0000 7f9167133640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9160162f10 con 0x7f916015e760 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.145+0000 7f9167133640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f9160162300 con 0x7f91600d0540 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f9164ecf640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f916015e760 0x7f91600d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f91656d0640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91600d1030 0x7f916009aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51906/0 (socket says 172.21.15.7:51906) 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f9164ecf640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f916015e760 0x7f91600d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54424/0 (socket says 172.21.15.7:54424) 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f91656d0640 1 -- 172.21.15.7:0/369174504 learned_addr learned my addr 172.21.15.7:0/369174504 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f9164ecf640 1 -- 172.21.15.7:0/369174504 >> v2:172.21.15.18:3300/0 conn(0x7f91600d0540 msgr2=0x7f91600d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f915ffff640 1 --2- 172.21.15.7:0/369174504 >> v2:172.21.15.18:3300/0 conn(0x7f91600d0540 0x7f91600d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f915f7fe640 1 -- 172.21.15.7:0/369174504 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 348139847 0 0) 0x7f9158001490 con 0x7f91600d1030 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f9164ecf640 1 --2- 172.21.15.7:0/369174504 >> v2:172.21.15.18:3300/0 conn(0x7f91600d0540 0x7f91600d0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f9164ecf640 1 -- 172.21.15.7:0/369174504 >> v1:172.21.15.7:6789/0 conn(0x7f91600d1030 legacy=0x7f916009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:17.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f9164ecf640 1 -- 172.21.15.7:0/369174504 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91601625e0 con 0x7f916015e760 2024-09-15T23:37:17.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.146+0000 7f915ffff640 1 --2- 172.21.15.7:0/369174504 >> v2:172.21.15.18:3300/0 conn(0x7f91600d0540 0x7f91600d0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:17.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f9164ecf640 1 --2- 172.21.15.7:0/369174504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916015e760 0x7f91600d0000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9160163260 tx=0x7f915002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b9e9515e233495 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:17.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f915f7fe640 1 -- 172.21.15.7:0/369174504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150037070 con 0x7f916015e760 2024-09-15T23:37:17.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f915f7fe640 1 -- 172.21.15.7:0/369174504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9150032440 con 0x7f916015e760 2024-09-15T23:37:17.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f915f7fe640 1 -- 172.21.15.7:0/369174504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150032810 con 0x7f916015e760 2024-09-15T23:37:17.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f9167133640 1 -- 172.21.15.7:0/369174504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916015e760 msgr2=0x7f91600d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:17.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f9167133640 1 --2- 172.21.15.7:0/369174504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916015e760 0x7f91600d0000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9160163260 tx=0x7f915002a1a0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f9167133640 1 -- 172.21.15.7:0/369174504 shutdown_connections 2024-09-15T23:37:17.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f9167133640 1 --2- 172.21.15.7:0/369174504 >> v2:172.21.15.18:3300/0 conn(0x7f91600d0540 0x7f91600d0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f9167133640 1 --2- 172.21.15.7:0/369174504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916015e760 0x7f91600d0000 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f9167133640 1 -- 172.21.15.7:0/369174504 >> 172.21.15.7:0/369174504 conn(0x7f91600d2430 msgr2=0x7f91600d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f9167133640 1 -- 172.21.15.7:0/369174504 shutdown_connections 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.147+0000 7f9167133640 1 -- 172.21.15.7:0/369174504 wait complete. 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.149+0000 7f9167133640 1 Processor -- start 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.149+0000 7f9167133640 1 -- start start 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.149+0000 7f9167133640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91600d0540 0x7f91601f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.149+0000 7f9167133640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91600d1030 0x7f9160156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9167133640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f916015e760 0x7f9160156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9167133640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91600ce4f0 con 0x7f91600d0540 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9167133640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9160162ba0 con 0x7f91600d1030 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9167133640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f91600ce670 con 0x7f916015e760 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9164ecf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91600d0540 0x7f91601f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:17.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9164ecf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91600d0540 0x7f91601f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54440/0 (socket says 172.21.15.7:54440) 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9164ecf640 1 -- 172.21.15.7:0/1137953607 learned_addr learned my addr 172.21.15.7:0/1137953607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f915ffff640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91600d1030 0x7f9160156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f91656d0640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f916015e760 0x7f9160156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9164ecf640 1 -- 172.21.15.7:0/1137953607 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f916015e760 msgr2=0x7f9160156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9164ecf640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f916015e760 0x7f9160156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9164ecf640 1 -- 172.21.15.7:0/1137953607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91600d1030 msgr2=0x7f9160156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9164ecf640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91600d1030 0x7f9160156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9164ecf640 1 -- 172.21.15.7:0/1137953607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91600c9a60 con 0x7f91600d0540 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9164ecf640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91600d0540 0x7f91601f4460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9150000c00 tx=0x7f91500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f915d7fa640 1 -- 172.21.15.7:0/1137953607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150037040 con 0x7f91600d0540 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f915d7fa640 1 -- 172.21.15.7:0/1137953607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9150042070 con 0x7f91600d0540 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f915d7fa640 1 -- 172.21.15.7:0/1137953607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150030a50 con 0x7f91600d0540 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91600c9d40 con 0x7f91600d0540 2024-09-15T23:37:17.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.150+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91600b63a0 con 0x7f91600d0540 2024-09-15T23:37:17.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.152+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91600d32d0 con 0x7f91600d0540 2024-09-15T23:37:17.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.152+0000 7f915d7fa640 1 -- 172.21.15.7:0/1137953607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9150030bf0 con 0x7f91600d0540 2024-09-15T23:37:17.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.153+0000 7f915d7fa640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f914006c2b0 0x7f914006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:17.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.153+0000 7f915d7fa640 1 -- 172.21.15.7:0/1137953607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f91500abef0 con 0x7f91600d0540 2024-09-15T23:37:17.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.153+0000 7f915ffff640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f914006c2b0 0x7f914006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:17.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.153+0000 7f915ffff640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f914006c2b0 0x7f914006e770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f91600d18e0 tx=0x7f9148008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:17.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.157+0000 7f915d7fa640 1 -- 172.21.15.7:0/1137953607 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9150079e80 con 0x7f91600d0540 2024-09-15T23:37:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:17.367+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9160160cf0 con 0x7f91600d0540 2024-09-15T23:37:18.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.032+0000 7f915d7fa640 1 -- 172.21.15.7:0/1137953607 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7f915007e180 con 0x7f91600d0540 2024-09-15T23:37:18.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f914006c2b0 msgr2=0x7f914006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f914006c2b0 0x7f914006e770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f91600d18e0 tx=0x7f9148008040 comp rx=0 tx=0).stop 2024-09-15T23:37:18.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91600d0540 msgr2=0x7f91601f4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91600d0540 0x7f91601f4460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9150000c00 tx=0x7f91500046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 shutdown_connections 2024-09-15T23:37:18.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f914006c2b0 0x7f914006e770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f916015e760 0x7f9160156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91600d1030 0x7f9160156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 --2- 172.21.15.7:0/1137953607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91600d0540 0x7f91601f4460 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 >> 172.21.15.7:0/1137953607 conn(0x7f91600d2430 msgr2=0x7f9160161210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:18.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 shutdown_connections 2024-09-15T23:37:18.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.036+0000 7f9167133640 1 -- 172.21.15.7:0/1137953607 wait complete. 2024-09-15T23:37:18.050 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-15T23:37:18.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.159+0000 7f8fc010a640 1 Processor -- start 2024-09-15T23:37:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.159+0000 7f8fc010a640 1 -- start start 2024-09-15T23:37:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fc010a640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8fb815e6d0 0x7f8fb80d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fc010a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb80d32a0 0x7f8fb80b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fc010a640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb80b3d10 con 0x7f8fb80d51c0 2024-09-15T23:37:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fc010a640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb80cfcd0 con 0x7f8fb815e6d0 2024-09-15T23:37:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fc010a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb80cff50 con 0x7f8fb80d32a0 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fbdea6640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8fb815e6d0 0x7f8fb80d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fbe6a7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb80d32a0 0x7f8fb80b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fbe6a7640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb80d32a0 0x7f8fb80b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:52430/0 (socket says 172.21.15.7:52430) 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fbd6a5640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f8fb80d51c0 0x7f8fb80d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:42850/0 (socket says 172.21.15.7:42850) 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fbdea6640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8fb815e6d0 0x7f8fb80d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:35000/0 (socket says 172.21.15.7:35000) 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.160+0000 7f8fbe6a7640 1 -- 172.21.15.7:0/3466173736 learned_addr learned my addr 172.21.15.7:0/3466173736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.161+0000 7f8fbe6a7640 1 -- 172.21.15.7:0/3466173736 >> v2:172.21.15.18:3300/0 conn(0x7f8fb815e6d0 msgr2=0x7f8fb80d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.161+0000 7f8fbe6a7640 1 --2- 172.21.15.7:0/3466173736 >> v2:172.21.15.18:3300/0 conn(0x7f8fb815e6d0 0x7f8fb80d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.161+0000 7f8fbe6a7640 1 -- 172.21.15.7:0/3466173736 >> v1:172.21.15.18:6789/0 conn(0x7f8fb80d51c0 legacy=0x7f8fb80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.161+0000 7f8fbe6a7640 1 -- 172.21.15.7:0/3466173736 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb80d0230 con 0x7f8fb80d32a0 2024-09-15T23:37:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fbe6a7640 1 --2- 172.21.15.7:0/3466173736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb80d32a0 0x7f8fb80b41c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009e50 tx=0x7f8fb002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=187bf84c03e504bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fbcea4640 1 -- 172.21.15.7:0/3466173736 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0036070 con 0x7f8fb80d32a0 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fbcea4640 1 -- 172.21.15.7:0/3466173736 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb0031600 con 0x7f8fb80d32a0 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fbcea4640 1 -- 172.21.15.7:0/3466173736 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00319d0 con 0x7f8fb80d32a0 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fc010a640 1 -- 172.21.15.7:0/3466173736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb80d32a0 msgr2=0x7f8fb80b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fc010a640 1 --2- 172.21.15.7:0/3466173736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb80d32a0 0x7f8fb80b41c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009e50 tx=0x7f8fb002a710 comp rx=0 tx=0).stop 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fc010a640 1 -- 172.21.15.7:0/3466173736 shutdown_connections 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fc010a640 1 --2- 172.21.15.7:0/3466173736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb80d32a0 0x7f8fb80b41c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fc010a640 1 --2- 172.21.15.7:0/3466173736 >> v2:172.21.15.18:3300/0 conn(0x7f8fb815e6d0 0x7f8fb80d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fc010a640 1 -- 172.21.15.7:0/3466173736 >> 172.21.15.7:0/3466173736 conn(0x7f8fb808bfd0 msgr2=0x7f8fb80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fc010a640 1 -- 172.21.15.7:0/3466173736 shutdown_connections 2024-09-15T23:37:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.162+0000 7f8fc010a640 1 -- 172.21.15.7:0/3466173736 wait complete. 2024-09-15T23:37:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.164+0000 7f8fc010a640 1 Processor -- start 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.164+0000 7f8fc010a640 1 -- start start 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.164+0000 7f8fc010a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb80d32a0 0x7f8fb8155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.164+0000 7f8fc010a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb80d51c0 0x7f8fb8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.164+0000 7f8fc010a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb815e6d0 0x7f8fb81610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.164+0000 7f8fc010a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb80d26f0 con 0x7f8fb80d51c0 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.164+0000 7f8fc010a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb80d0510 con 0x7f8fb80d32a0 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.164+0000 7f8fc010a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb80d2870 con 0x7f8fb815e6d0 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbd6a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb80d51c0 0x7f8fb8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbd6a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb80d51c0 0x7f8fb8160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54452/0 (socket says 172.21.15.7:54452) 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbd6a5640 1 -- 172.21.15.7:0/2759526769 learned_addr learned my addr 172.21.15.7:0/2759526769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbdea6640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb80d32a0 0x7f8fb8155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbe6a7640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb815e6d0 0x7f8fb81610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbd6a5640 1 -- 172.21.15.7:0/2759526769 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb815e6d0 msgr2=0x7f8fb81610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbd6a5640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb815e6d0 0x7f8fb81610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbd6a5640 1 -- 172.21.15.7:0/2759526769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb80d32a0 msgr2=0x7f8fb8155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbd6a5640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb80d32a0 0x7f8fb8155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbd6a5640 1 -- 172.21.15.7:0/2759526769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb815ee70 con 0x7f8fb80d51c0 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fbd6a5640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb80d51c0 0x7f8fb8160b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800c440 tx=0x7f8fa8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fae7fc640 1 -- 172.21.15.7:0/2759526769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80074e0 con 0x7f8fb80d51c0 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fae7fc640 1 -- 172.21.15.7:0/2759526769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa8007680 con 0x7f8fb80d51c0 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb815fbf0 con 0x7f8fb80d51c0 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fae7fc640 1 -- 172.21.15.7:0/2759526769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8004a20 con 0x7f8fb80d51c0 2024-09-15T23:37:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.165+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb815f3f0 con 0x7f8fb80d51c0 2024-09-15T23:37:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.166+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb80d21b0 con 0x7f8fb80d51c0 2024-09-15T23:37:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.167+0000 7f8fae7fc640 1 -- 172.21.15.7:0/2759526769 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8fa801a020 con 0x7f8fb80d51c0 2024-09-15T23:37:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.167+0000 7f8fae7fc640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8f9c06c2b0 0x7f8f9c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.167+0000 7f8fae7fc640 1 -- 172.21.15.7:0/2759526769 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f8fa808ad50 con 0x7f8fb80d51c0 2024-09-15T23:37:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.167+0000 7f8fbdea6640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8f9c06c2b0 0x7f8f9c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.168+0000 7f8fbdea6640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8f9c06c2b0 0x7f8f9c06e770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80cb4a0 tx=0x7f8fb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.172+0000 7f8fae7fc640 1 -- 172.21.15.7:0/2759526769 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8fa808e030 con 0x7f8fb80d51c0 2024-09-15T23:37:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.381+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fb80d1cb0 con 0x7f8fb80d51c0 2024-09-15T23:37:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.381+0000 7f8fae7fc640 1 -- 172.21.15.7:0/2759526769 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5214 (secure 0 0 0) 0x7f8fa8010300 con 0x7f8fb80d51c0 2024-09-15T23:37:18.382 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"btime":"2024-09-15T23:37:18:028501+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1101755994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6833","nonce":1101755994},{"type":"v1","addr":"172.21.15.18:6835","nonce":1101755994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6834/2816400413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2816400413},{"type":"v1","addr":"172.21.15.18:6834","nonce":2816400413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4827,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/809864756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":809864756},{"type":"v1","addr":"172.21.15.7:6836","nonce":809864756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:37:15.374444+0000","modified":"2024-09-15T23:37:17.368915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4830},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4830":{"gid":4830,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/1729414280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1729414280},{"type":"v1","addr":"172.21.15.7:6837","nonce":1729414280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4830,"qdb_cluster":[4830]},"id":2}]} 2024-09-15T23:37:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 15 2024-09-15T23:37:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8f9c06c2b0 msgr2=0x7f8f9c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8f9c06c2b0 0x7f8f9c06e770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80cb4a0 tx=0x7f8fb4008040 comp rx=0 tx=0).stop 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb80d51c0 msgr2=0x7f8fb8160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb80d51c0 0x7f8fb8160b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800c440 tx=0x7f8fa8001e70 comp rx=0 tx=0).stop 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 shutdown_connections 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8f9c06c2b0 0x7f8f9c06e770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb815e6d0 0x7f8fb81610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb80d51c0 0x7f8fb8160b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 --2- 172.21.15.7:0/2759526769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb80d32a0 0x7f8fb8155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.383+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 >> 172.21.15.7:0/2759526769 conn(0x7f8fb808bfd0 msgr2=0x7f8fb806d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.384+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 shutdown_connections 2024-09-15T23:37:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.384+0000 7f8fc010a640 1 -- 172.21.15.7:0/2759526769 wait complete. 2024-09-15T23:37:18.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:37:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.497+0000 7f650ed6d640 1 Processor -- start 2024-09-15T23:37:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.497+0000 7f650ed6d640 1 -- start start 2024-09-15T23:37:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.497+0000 7f650ed6d640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f65080d52e0 0x7f650806d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.497+0000 7f650ed6d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f650806dbb0 0x7f65080d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.497+0000 7f650ed6d640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65080d0a30 con 0x7f650815e840 2024-09-15T23:37:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.497+0000 7f650ed6d640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f65080cfe90 con 0x7f650806dbb0 2024-09-15T23:37:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.497+0000 7f650ed6d640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f65080cf560 con 0x7f65080d52e0 2024-09-15T23:37:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.498+0000 7f650d30a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f650806dbb0 0x7f65080d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.498+0000 7f650d30a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f650806dbb0 0x7f65080d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54466/0 (socket says 172.21.15.7:54466) 2024-09-15T23:37:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.498+0000 7f650cb09640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f650815e840 0x7f65080d0320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51914/0 (socket says 172.21.15.7:51914) 2024-09-15T23:37:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.498+0000 7f650d30a640 1 -- 172.21.15.7:0/3994209066 learned_addr learned my addr 172.21.15.7:0/3994209066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.498+0000 7f650d30a640 1 -- 172.21.15.7:0/3994209066 >> v2:172.21.15.18:3300/0 conn(0x7f65080d52e0 msgr2=0x7f650806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.498+0000 7f650d30a640 1 --2- 172.21.15.7:0/3994209066 >> v2:172.21.15.18:3300/0 conn(0x7f65080d52e0 0x7f650806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.498+0000 7f650d30a640 1 -- 172.21.15.7:0/3994209066 >> v1:172.21.15.7:6789/0 conn(0x7f650815e840 legacy=0x7f65080d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.498+0000 7f65077fe640 1 -- 172.21.15.7:0/3994209066 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3149236308 0 0) 0x7f65080d0a30 con 0x7f650815e840 2024-09-15T23:37:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.498+0000 7f650d30a640 1 -- 172.21.15.7:0/3994209066 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65080cf840 con 0x7f650806dbb0 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f650d30a640 1 --2- 172.21.15.7:0/3994209066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 0x7f65080d0f90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f65080c7e60 tx=0x7f650002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e410c400813f36d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f65077fe640 1 -- 172.21.15.7:0/3994209066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500037070 con 0x7f650806dbb0 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f65077fe640 1 -- 172.21.15.7:0/3994209066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6500032440 con 0x7f650806dbb0 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f65077fe640 1 -- 172.21.15.7:0/3994209066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500032810 con 0x7f650806dbb0 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f650ed6d640 1 -- 172.21.15.7:0/3994209066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 msgr2=0x7f65080d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f650ed6d640 1 --2- 172.21.15.7:0/3994209066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 0x7f65080d0f90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f65080c7e60 tx=0x7f650002a270 comp rx=0 tx=0).stop 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f650ed6d640 1 -- 172.21.15.7:0/3994209066 shutdown_connections 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f650ed6d640 1 --2- 172.21.15.7:0/3994209066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 0x7f65080d0f90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f650ed6d640 1 --2- 172.21.15.7:0/3994209066 >> v2:172.21.15.18:3300/0 conn(0x7f65080d52e0 0x7f650806d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f650ed6d640 1 -- 172.21.15.7:0/3994209066 >> 172.21.15.7:0/3994209066 conn(0x7f650808bfd0 msgr2=0x7f65080d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f650ed6d640 1 -- 172.21.15.7:0/3994209066 shutdown_connections 2024-09-15T23:37:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.499+0000 7f650ed6d640 1 -- 172.21.15.7:0/3994209066 wait complete. 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.501+0000 7f650ed6d640 1 Processor -- start 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.501+0000 7f650ed6d640 1 -- start start 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650ed6d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 0x7f65080b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650ed6d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65080d52e0 0x7f65080ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650ed6d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f650815e840 0x7f6508155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650ed6d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f650815ec20 con 0x7f650806dbb0 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650ed6d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f650806e030 con 0x7f650815e840 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650ed6d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f65080b4a30 con 0x7f65080d52e0 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650cb09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 0x7f65080b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650cb09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 0x7f65080b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54476/0 (socket says 172.21.15.7:54476) 2024-09-15T23:37:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650cb09640 1 -- 172.21.15.7:0/4069852825 learned_addr learned my addr 172.21.15.7:0/4069852825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650d30a640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f650815e840 0x7f6508155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f6507fff640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65080d52e0 0x7f65080ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650cb09640 1 -- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65080d52e0 msgr2=0x7f65080ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650cb09640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65080d52e0 0x7f65080ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650cb09640 1 -- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f650815e840 msgr2=0x7f6508155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650cb09640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f650815e840 0x7f6508155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650cb09640 1 -- 172.21.15.7:0/4069852825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65080ca590 con 0x7f650806dbb0 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f6507fff640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65080d52e0 0x7f65080ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650d30a640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f650815e840 0x7f6508155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650cb09640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 0x7f65080b5d20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f64f000c430 tx=0x7f64f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f65057fa640 1 -- 172.21.15.7:0/4069852825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f00074e0 con 0x7f650806dbb0 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f65057fa640 1 -- 172.21.15.7:0/4069852825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f00047e0 con 0x7f650806dbb0 2024-09-15T23:37:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65080b4c80 con 0x7f650806dbb0 2024-09-15T23:37:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f65057fa640 1 -- 172.21.15.7:0/4069852825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0004bd0 con 0x7f650806dbb0 2024-09-15T23:37:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.502+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65080b5240 con 0x7f650806dbb0 2024-09-15T23:37:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.503+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650806d670 con 0x7f650806dbb0 2024-09-15T23:37:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.504+0000 7f65057fa640 1 -- 172.21.15.7:0/4069852825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f64f0011070 con 0x7f650806dbb0 2024-09-15T23:37:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.504+0000 7f65057fa640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64e806c2b0 0x7f64e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.504+0000 7f65057fa640 1 -- 172.21.15.7:0/4069852825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f64f008b220 con 0x7f650806dbb0 2024-09-15T23:37:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.504+0000 7f6507fff640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64e806c2b0 0x7f64e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.505+0000 7f6507fff640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64e806c2b0 0x7f64e806e770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f65080cb300 tx=0x7f64f80073c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.508+0000 7f65057fa640 1 -- 172.21.15.7:0/4069852825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f64f000e280 con 0x7f650806dbb0 2024-09-15T23:37:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.687+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65080b2660 con 0x7f650806dbb0 2024-09-15T23:37:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.688+0000 7f65057fa640 1 -- 172.21.15.7:0/4069852825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16906 (secure 0 0 0) 0x7f64f0059800 con 0x7f650806dbb0 2024-09-15T23:37:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:18.689 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":39,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:37:17.023842+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:37:13.310698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-15T23:37:14.361918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:37:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64e806c2b0 msgr2=0x7f64e806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64e806c2b0 0x7f64e806e770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f65080cb300 tx=0x7f64f80073c0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 msgr2=0x7f65080b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 0x7f65080b5d20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f64f000c430 tx=0x7f64f000c8f0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 shutdown_connections 2024-09-15T23:37:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f650815e840 0x7f6508155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f64e806c2b0 0x7f64e806e770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65080d52e0 0x7f65080ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 --2- 172.21.15.7:0/4069852825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650806dbb0 0x7f65080b5d20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 >> 172.21.15.7:0/4069852825 conn(0x7f650808bfd0 msgr2=0x7f65080b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 shutdown_connections 2024-09-15T23:37:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.690+0000 7f650ed6d640 1 -- 172.21.15.7:0/4069852825 wait complete. 2024-09-15T23:37:18.702 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T23:37:18.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:37:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.804+0000 7f6badadd640 1 Processor -- start 2024-09-15T23:37:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.804+0000 7f6badadd640 1 -- start start 2024-09-15T23:37:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6badadd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ba815e6d0 0x7f6ba80d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6badadd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6ba80d32a0 0x7f6ba80b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6badadd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba80b3d10 con 0x7f6ba80d51c0 2024-09-15T23:37:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6badadd640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80cfcd0 con 0x7f6ba815e6d0 2024-09-15T23:37:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6badadd640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80cff50 con 0x7f6ba80d32a0 2024-09-15T23:37:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba77fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ba815e6d0 0x7f6ba80d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba6ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6ba80d51c0 0x7f6ba80d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51928/0 (socket says 172.21.15.7:51928) 2024-09-15T23:37:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba6ffd640 1 -- 172.21.15.7:0/1438980195 learned_addr learned my addr 172.21.15.7:0/1438980195 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba7fff640 1 --2- 172.21.15.7:0/1438980195 >> v2:172.21.15.18:3300/0 conn(0x7f6ba80d32a0 0x7f6ba80b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba77fe640 1 -- 172.21.15.7:0/1438980195 >> v2:172.21.15.18:3300/0 conn(0x7f6ba80d32a0 msgr2=0x7f6ba80b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba77fe640 1 --2- 172.21.15.7:0/1438980195 >> v2:172.21.15.18:3300/0 conn(0x7f6ba80d32a0 0x7f6ba80b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba77fe640 1 -- 172.21.15.7:0/1438980195 >> v1:172.21.15.7:6789/0 conn(0x7f6ba80d51c0 legacy=0x7f6ba80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba77fe640 1 -- 172.21.15.7:0/1438980195 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80d0230 con 0x7f6ba815e6d0 2024-09-15T23:37:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba67fc640 1 -- 172.21.15.7:0/1438980195 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3610194606 0 0) 0x7f6ba80b3d10 con 0x7f6ba80d51c0 2024-09-15T23:37:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.805+0000 7f6ba7fff640 1 --2- 172.21.15.7:0/1438980195 >> v2:172.21.15.18:3300/0 conn(0x7f6ba80d32a0 0x7f6ba80b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.806+0000 7f6ba77fe640 1 --2- 172.21.15.7:0/1438980195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba815e6d0 0x7f6ba80d21b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80d3800 tx=0x7f6b9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd4458c52f3626a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.806+0000 7f6ba67fc640 1 -- 172.21.15.7:0/1438980195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037070 con 0x7f6ba815e6d0 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.806+0000 7f6ba67fc640 1 -- 172.21.15.7:0/1438980195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98032440 con 0x7f6ba815e6d0 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.806+0000 7f6ba67fc640 1 -- 172.21.15.7:0/1438980195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98032810 con 0x7f6ba815e6d0 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.807+0000 7f6badadd640 1 -- 172.21.15.7:0/1438980195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba815e6d0 msgr2=0x7f6ba80d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.807+0000 7f6badadd640 1 --2- 172.21.15.7:0/1438980195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba815e6d0 0x7f6ba80d21b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80d3800 tx=0x7f6b9802a1a0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.807+0000 7f6badadd640 1 -- 172.21.15.7:0/1438980195 shutdown_connections 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.807+0000 7f6badadd640 1 --2- 172.21.15.7:0/1438980195 >> v2:172.21.15.18:3300/0 conn(0x7f6ba80d32a0 0x7f6ba80b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.807+0000 7f6badadd640 1 --2- 172.21.15.7:0/1438980195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba815e6d0 0x7f6ba80d21b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.807+0000 7f6badadd640 1 -- 172.21.15.7:0/1438980195 >> 172.21.15.7:0/1438980195 conn(0x7f6ba808bfd0 msgr2=0x7f6ba80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.807+0000 7f6badadd640 1 -- 172.21.15.7:0/1438980195 shutdown_connections 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.807+0000 7f6badadd640 1 -- 172.21.15.7:0/1438980195 wait complete. 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.808+0000 7f6badadd640 1 Processor -- start 2024-09-15T23:37:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.808+0000 7f6badadd640 1 -- start start 2024-09-15T23:37:18.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6badadd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba80d32a0 0x7f6ba80c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6badadd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba80d51c0 0x7f6ba80cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6badadd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ba815e6d0 0x7f6ba80d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6badadd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba80d26f0 con 0x7f6ba80d32a0 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6badadd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba80d0510 con 0x7f6ba80d51c0 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6badadd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba80d2870 con 0x7f6ba815e6d0 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba80d32a0 0x7f6ba80c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba80d32a0 0x7f6ba80c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54490/0 (socket says 172.21.15.7:54490) 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba77fe640 1 -- 172.21.15.7:0/2784797169 learned_addr learned my addr 172.21.15.7:0/2784797169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba7fff640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ba815e6d0 0x7f6ba80d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba77fe640 1 -- 172.21.15.7:0/2784797169 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ba815e6d0 msgr2=0x7f6ba80d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba77fe640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ba815e6d0 0x7f6ba80d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba6ffd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba80d51c0 0x7f6ba80cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba77fe640 1 -- 172.21.15.7:0/2784797169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba80d51c0 msgr2=0x7f6ba80cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba77fe640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba80d51c0 0x7f6ba80cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba77fe640 1 -- 172.21.15.7:0/2784797169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80cdfd0 con 0x7f6ba80d32a0 2024-09-15T23:37:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba6ffd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba80d51c0 0x7f6ba80cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6ba77fe640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba80d32a0 0x7f6ba80c82f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.809+0000 7f6b8ffff640 1 -- 172.21.15.7:0/2784797169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037040 con 0x7f6ba80d32a0 2024-09-15T23:37:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.810+0000 7f6b8ffff640 1 -- 172.21.15.7:0/2784797169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98042070 con 0x7f6ba80d32a0 2024-09-15T23:37:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.810+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba80cd730 con 0x7f6ba80d32a0 2024-09-15T23:37:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.810+0000 7f6b8ffff640 1 -- 172.21.15.7:0/2784797169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98030bd0 con 0x7f6ba80d32a0 2024-09-15T23:37:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.810+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba80cdcf0 con 0x7f6ba80d32a0 2024-09-15T23:37:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.810+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba815efa0 con 0x7f6ba80d32a0 2024-09-15T23:37:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.811+0000 7f6b8ffff640 1 -- 172.21.15.7:0/2784797169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6b98047460 con 0x7f6ba80d32a0 2024-09-15T23:37:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.811+0000 7f6b8ffff640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6b9406c2b0 0x7f6b9406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.812+0000 7f6b8ffff640 1 -- 172.21.15.7:0/2784797169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6b980ac1a0 con 0x7f6ba80d32a0 2024-09-15T23:37:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.812+0000 7f6ba6ffd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6b9406c2b0 0x7f6b9406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.812+0000 7f6ba6ffd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6b9406c2b0 0x7f6b9406e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ba81f7980 tx=0x7f6b9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:18.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.816+0000 7f6b8ffff640 1 -- 172.21.15.7:0/2784797169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b98032e20 con 0x7f6ba80d32a0 2024-09-15T23:37:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.986+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ba80b41c0 con 0x7f6b9406c2b0 2024-09-15T23:37:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.992+0000 7f6b8ffff640 1 -- 172.21.15.7:0/2784797169 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247699 (secure 0 0 0) 0x7f6ba80b41c0 con 0x7f6b9406c2b0 2024-09-15T23:37:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:37:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6b9406c2b0 msgr2=0x7f6b9406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6b9406c2b0 0x7f6b9406e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ba81f7980 tx=0x7f6b9c009040 comp rx=0 tx=0).stop 2024-09-15T23:37:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba80d32a0 msgr2=0x7f6ba80c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba80d32a0 0x7f6ba80c82f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b980046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 shutdown_connections 2024-09-15T23:37:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6b9406c2b0 0x7f6b9406e770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6ba815e6d0 0x7f6ba80d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba80d51c0 0x7f6ba80cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 --2- 172.21.15.7:0/2784797169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba80d32a0 0x7f6ba80c82f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 >> 172.21.15.7:0/2784797169 conn(0x7f6ba808bfd0 msgr2=0x7f6ba806ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 shutdown_connections 2024-09-15T23:37:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:18.995+0000 7f6badadd640 1 -- 172.21.15.7:0/2784797169 wait complete. 2024-09-15T23:37:19.006 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-15T23:37:17.823024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254004,"kb_used_data":4476,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720716,"statfs":{"total":773094113280,"available":772834013184,"internally_reserved":0,"allocated":4583424,"data_stored":2621098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000643"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014716+0000","last_change":"2024-09-15T23:37:15.014716+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014716+0000","last_undegraded":"2024-09-15T23:37:15.014716+0000","last_fullsized":"2024-09-15T23:37:15.014716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013817+0000","last_change":"2024-09-15T23:37:15.013817+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013817+0000","last_undegraded":"2024-09-15T23:37:15.013817+0000","last_fullsized":"2024-09-15T23:37:15.013817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015854+0000","last_change":"2024-09-15T23:37:15.015854+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015854+0000","last_undegraded":"2024-09-15T23:37:15.015854+0000","last_fullsized":"2024-09-15T23:37:15.015854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048204+0000","last_change":"2024-09-15T23:37:16.039624+0000","last_active":"2024-09-15T23:37:16.048204+0000","last_peered":"2024-09-15T23:37:16.048204+0000","last_clean":"2024-09-15T23:37:16.048204+0000","last_became_active":"2024-09-15T23:37:16.038879+0000","last_became_peered":"2024-09-15T23:37:16.038879+0000","last_unstale":"2024-09-15T23:37:16.048204+0000","last_undegraded":"2024-09-15T23:37:16.048204+0000","last_fullsized":"2024-09-15T23:37:16.048204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:51.545065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013261+0000","last_change":"2024-09-15T23:37:15.013261+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013261+0000","last_undegraded":"2024-09-15T23:37:15.013261+0000","last_fullsized":"2024-09-15T23:37:15.013261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051987+0000","last_change":"2024-09-15T23:37:16.043510+0000","last_active":"2024-09-15T23:37:16.051987+0000","last_peered":"2024-09-15T23:37:16.051987+0000","last_clean":"2024-09-15T23:37:16.051987+0000","last_became_active":"2024-09-15T23:37:16.043208+0000","last_became_peered":"2024-09-15T23:37:16.043208+0000","last_unstale":"2024-09-15T23:37:16.051987+0000","last_undegraded":"2024-09-15T23:37:16.051987+0000","last_fullsized":"2024-09-15T23:37:16.051987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:10.099889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052859+0000","last_change":"2024-09-15T23:37:16.050348+0000","last_active":"2024-09-15T23:37:16.052859+0000","last_peered":"2024-09-15T23:37:16.052859+0000","last_clean":"2024-09-15T23:37:16.052859+0000","last_became_active":"2024-09-15T23:37:16.049871+0000","last_became_peered":"2024-09-15T23:37:16.049871+0000","last_unstale":"2024-09-15T23:37:16.052859+0000","last_undegraded":"2024-09-15T23:37:16.052859+0000","last_fullsized":"2024-09-15T23:37:16.052859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:23.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015421+0000","last_change":"2024-09-15T23:37:15.015421+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015421+0000","last_undegraded":"2024-09-15T23:37:15.015421+0000","last_fullsized":"2024-09-15T23:37:15.015421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047271+0000","last_change":"2024-09-15T23:37:16.037928+0000","last_active":"2024-09-15T23:37:16.047271+0000","last_peered":"2024-09-15T23:37:16.047271+0000","last_clean":"2024-09-15T23:37:16.047271+0000","last_became_active":"2024-09-15T23:37:16.037645+0000","last_became_peered":"2024-09-15T23:37:16.037645+0000","last_unstale":"2024-09-15T23:37:16.047271+0000","last_undegraded":"2024-09-15T23:37:16.047271+0000","last_fullsized":"2024-09-15T23:37:16.047271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:29.660110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048427+0000","last_change":"2024-09-15T23:37:16.046081+0000","last_active":"2024-09-15T23:37:16.048427+0000","last_peered":"2024-09-15T23:37:16.048427+0000","last_clean":"2024-09-15T23:37:16.048427+0000","last_became_active":"2024-09-15T23:37:16.045733+0000","last_became_peered":"2024-09-15T23:37:16.045733+0000","last_unstale":"2024-09-15T23:37:16.048427+0000","last_undegraded":"2024-09-15T23:37:16.048427+0000","last_fullsized":"2024-09-15T23:37:16.048427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036156+0000","last_change":"2024-09-15T23:37:16.024098+0000","last_active":"2024-09-15T23:37:16.036156+0000","last_peered":"2024-09-15T23:37:16.036156+0000","last_clean":"2024-09-15T23:37:16.036156+0000","last_became_active":"2024-09-15T23:37:16.023753+0000","last_became_peered":"2024-09-15T23:37:16.023753+0000","last_unstale":"2024-09-15T23:37:16.036156+0000","last_undegraded":"2024-09-15T23:37:16.036156+0000","last_fullsized":"2024-09-15T23:37:16.036156+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:34.586227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052091+0000","last_change":"2024-09-15T23:37:16.048043+0000","last_active":"2024-09-15T23:37:16.052091+0000","last_peered":"2024-09-15T23:37:16.052091+0000","last_clean":"2024-09-15T23:37:16.052091+0000","last_became_active":"2024-09-15T23:37:16.047587+0000","last_became_peered":"2024-09-15T23:37:16.047587+0000","last_unstale":"2024-09-15T23:37:16.052091+0000","last_undegraded":"2024-09-15T23:37:16.052091+0000","last_fullsized":"2024-09-15T23:37:16.052091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:57.383248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035550+0000","last_change":"2024-09-15T23:37:16.034026+0000","last_active":"2024-09-15T23:37:16.035550+0000","last_peered":"2024-09-15T23:37:16.035550+0000","last_clean":"2024-09-15T23:37:16.035550+0000","last_became_active":"2024-09-15T23:37:16.033385+0000","last_became_peered":"2024-09-15T23:37:16.033385+0000","last_unstale":"2024-09-15T23:37:16.035550+0000","last_undegraded":"2024-09-15T23:37:16.035550+0000","last_fullsized":"2024-09-15T23:37:16.035550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:33.923553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035623+0000","last_change":"2024-09-15T23:37:16.034152+0000","last_active":"2024-09-15T23:37:16.035623+0000","last_peered":"2024-09-15T23:37:16.035623+0000","last_clean":"2024-09-15T23:37:16.035623+0000","last_became_active":"2024-09-15T23:37:16.029607+0000","last_became_peered":"2024-09-15T23:37:16.029607+0000","last_unstale":"2024-09-15T23:37:16.035623+0000","last_undegraded":"2024-09-15T23:37:16.035623+0000","last_fullsized":"2024-09-15T23:37:16.035623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:55.516820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042577+0000","last_change":"2024-09-15T23:37:16.037756+0000","last_active":"2024-09-15T23:37:16.042577+0000","last_peered":"2024-09-15T23:37:16.042577+0000","last_clean":"2024-09-15T23:37:16.042577+0000","last_became_active":"2024-09-15T23:37:16.037309+0000","last_became_peered":"2024-09-15T23:37:16.037309+0000","last_unstale":"2024-09-15T23:37:16.042577+0000","last_undegraded":"2024-09-15T23:37:16.042577+0000","last_fullsized":"2024-09-15T23:37:16.042577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:45.986835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042641+0000","last_change":"2024-09-15T23:37:16.037039+0000","last_active":"2024-09-15T23:37:16.042641+0000","last_peered":"2024-09-15T23:37:16.042641+0000","last_clean":"2024-09-15T23:37:16.042641+0000","last_became_active":"2024-09-15T23:37:16.036774+0000","last_became_peered":"2024-09-15T23:37:16.036774+0000","last_unstale":"2024-09-15T23:37:16.042641+0000","last_undegraded":"2024-09-15T23:37:16.042641+0000","last_fullsized":"2024-09-15T23:37:16.042641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:30.804394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.040576+0000","last_change":"2024-09-15T23:37:16.034353+0000","last_active":"2024-09-15T23:37:16.040576+0000","last_peered":"2024-09-15T23:37:16.040576+0000","last_clean":"2024-09-15T23:37:16.040576+0000","last_became_active":"2024-09-15T23:37:16.033979+0000","last_became_peered":"2024-09-15T23:37:16.033979+0000","last_unstale":"2024-09-15T23:37:16.040576+0000","last_undegraded":"2024-09-15T23:37:16.040576+0000","last_fullsized":"2024-09-15T23:37:16.040576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:05.128368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036134+0000","last_change":"2024-09-15T23:37:16.027056+0000","last_active":"2024-09-15T23:37:16.036134+0000","last_peered":"2024-09-15T23:37:16.036134+0000","last_clean":"2024-09-15T23:37:16.036134+0000","last_became_active":"2024-09-15T23:37:16.026780+0000","last_became_peered":"2024-09-15T23:37:16.026780+0000","last_unstale":"2024-09-15T23:37:16.036134+0000","last_undegraded":"2024-09-15T23:37:16.036134+0000","last_fullsized":"2024-09-15T23:37:16.036134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:43.279890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044498+0000","last_change":"2024-09-15T23:37:16.042298+0000","last_active":"2024-09-15T23:37:16.044498+0000","last_peered":"2024-09-15T23:37:16.044498+0000","last_clean":"2024-09-15T23:37:16.044498+0000","last_became_active":"2024-09-15T23:37:16.041961+0000","last_became_peered":"2024-09-15T23:37:16.041961+0000","last_unstale":"2024-09-15T23:37:16.044498+0000","last_undegraded":"2024-09-15T23:37:16.044498+0000","last_fullsized":"2024-09-15T23:37:16.044498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:14.340846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033699+0000","last_change":"2024-09-15T23:37:16.031292+0000","last_active":"2024-09-15T23:37:16.033699+0000","last_peered":"2024-09-15T23:37:16.033699+0000","last_clean":"2024-09-15T23:37:16.033699+0000","last_became_active":"2024-09-15T23:37:16.030975+0000","last_became_peered":"2024-09-15T23:37:16.030975+0000","last_unstale":"2024-09-15T23:37:16.033699+0000","last_undegraded":"2024-09-15T23:37:16.033699+0000","last_fullsized":"2024-09-15T23:37:16.033699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:12.360351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048310+0000","last_change":"2024-09-15T23:37:16.039927+0000","last_active":"2024-09-15T23:37:16.048310+0000","last_peered":"2024-09-15T23:37:16.048310+0000","last_clean":"2024-09-15T23:37:16.048310+0000","last_became_active":"2024-09-15T23:37:16.039622+0000","last_became_peered":"2024-09-15T23:37:16.039622+0000","last_unstale":"2024-09-15T23:37:16.048310+0000","last_undegraded":"2024-09-15T23:37:16.048310+0000","last_fullsized":"2024-09-15T23:37:16.048310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:54.978065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035781+0000","last_change":"2024-09-15T23:37:16.032318+0000","last_active":"2024-09-15T23:37:16.035781+0000","last_peered":"2024-09-15T23:37:16.035781+0000","last_clean":"2024-09-15T23:37:16.035781+0000","last_became_active":"2024-09-15T23:37:16.028787+0000","last_became_peered":"2024-09-15T23:37:16.028787+0000","last_unstale":"2024-09-15T23:37:16.035781+0000","last_undegraded":"2024-09-15T23:37:16.035781+0000","last_fullsized":"2024-09-15T23:37:16.035781+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:31.065835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035685+0000","last_change":"2024-09-15T23:37:16.028365+0000","last_active":"2024-09-15T23:37:16.035685+0000","last_peered":"2024-09-15T23:37:16.035685+0000","last_clean":"2024-09-15T23:37:16.035685+0000","last_became_active":"2024-09-15T23:37:16.028051+0000","last_became_peered":"2024-09-15T23:37:16.028051+0000","last_unstale":"2024-09-15T23:37:16.035685+0000","last_undegraded":"2024-09-15T23:37:16.035685+0000","last_fullsized":"2024-09-15T23:37:16.035685+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:47.439397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048225+0000","last_change":"2024-09-15T23:37:16.039870+0000","last_active":"2024-09-15T23:37:16.048225+0000","last_peered":"2024-09-15T23:37:16.048225+0000","last_clean":"2024-09-15T23:37:16.048225+0000","last_became_active":"2024-09-15T23:37:16.039295+0000","last_became_peered":"2024-09-15T23:37:16.039295+0000","last_unstale":"2024-09-15T23:37:16.048225+0000","last_undegraded":"2024-09-15T23:37:16.048225+0000","last_fullsized":"2024-09-15T23:37:16.048225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:16.959191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046178+0000","last_change":"2024-09-15T23:37:16.044114+0000","last_active":"2024-09-15T23:37:16.046178+0000","last_peered":"2024-09-15T23:37:16.046178+0000","last_clean":"2024-09-15T23:37:16.046178+0000","last_became_active":"2024-09-15T23:37:16.042005+0000","last_became_peered":"2024-09-15T23:37:16.042005+0000","last_unstale":"2024-09-15T23:37:16.046178+0000","last_undegraded":"2024-09-15T23:37:16.046178+0000","last_fullsized":"2024-09-15T23:37:16.046178+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:40.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056140+0000","last_change":"2024-09-15T23:37:15.019375+0000","last_active":"2024-09-15T23:37:16.056140+0000","last_peered":"2024-09-15T23:37:16.056140+0000","last_clean":"2024-09-15T23:37:16.056140+0000","last_became_active":"2024-09-15T23:37:15.018977+0000","last_became_peered":"2024-09-15T23:37:15.018977+0000","last_unstale":"2024-09-15T23:37:16.056140+0000","last_undegraded":"2024-09-15T23:37:16.056140+0000","last_fullsized":"2024-09-15T23:37:16.056140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:02.654673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013153+0000","last_change":"2024-09-15T23:37:15.013153+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013153+0000","last_undegraded":"2024-09-15T23:37:15.013153+0000","last_fullsized":"2024-09-15T23:37:15.013153+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.027944+0000","last_change":"2024-09-15T23:37:15.024812+0000","last_active":"2024-09-15T23:37:15.027944+0000","last_peered":"2024-09-15T23:37:15.027944+0000","last_clean":"2024-09-15T23:37:15.027944+0000","last_became_active":"2024-09-15T23:37:15.024533+0000","last_became_peered":"2024-09-15T23:37:15.024533+0000","last_unstale":"2024-09-15T23:37:15.027944+0000","last_undegraded":"2024-09-15T23:37:15.027944+0000","last_fullsized":"2024-09-15T23:37:15.027944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:56.378313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046817+0000","last_change":"2024-09-15T23:37:16.043209+0000","last_active":"2024-09-15T23:37:16.046817+0000","last_peered":"2024-09-15T23:37:16.046817+0000","last_clean":"2024-09-15T23:37:16.046817+0000","last_became_active":"2024-09-15T23:37:16.042016+0000","last_became_peered":"2024-09-15T23:37:16.042016+0000","last_unstale":"2024-09-15T23:37:16.046817+0000","last_undegraded":"2024-09-15T23:37:16.046817+0000","last_fullsized":"2024-09-15T23:37:16.046817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:17.167815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008364+0000","last_change":"2024-09-15T23:37:14.008364+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008364+0000","last_undegraded":"2024-09-15T23:37:14.008364+0000","last_fullsized":"2024-09-15T23:37:14.008364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014181+0000","last_change":"2024-09-15T23:37:15.014181+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014181+0000","last_undegraded":"2024-09-15T23:37:15.014181+0000","last_fullsized":"2024-09-15T23:37:15.014181+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056405+0000","last_change":"2024-09-15T23:37:15.023109+0000","last_active":"2024-09-15T23:37:16.056405+0000","last_peered":"2024-09-15T23:37:16.056405+0000","last_clean":"2024-09-15T23:37:16.056405+0000","last_became_active":"2024-09-15T23:37:15.022494+0000","last_became_peered":"2024-09-15T23:37:15.022494+0000","last_unstale":"2024-09-15T23:37:16.056405+0000","last_undegraded":"2024-09-15T23:37:16.056405+0000","last_fullsized":"2024-09-15T23:37:16.056405+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:29.332641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.055917+0000","last_change":"2024-09-15T23:37:15.025209+0000","last_active":"2024-09-15T23:37:16.055917+0000","last_peered":"2024-09-15T23:37:16.055917+0000","last_clean":"2024-09-15T23:37:16.055917+0000","last_became_active":"2024-09-15T23:37:15.022678+0000","last_became_peered":"2024-09-15T23:37:15.022678+0000","last_unstale":"2024-09-15T23:37:16.055917+0000","last_undegraded":"2024-09-15T23:37:16.055917+0000","last_fullsized":"2024-09-15T23:37:16.055917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:01.000785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033653+0000","last_change":"2024-09-15T23:37:16.031990+0000","last_active":"2024-09-15T23:37:16.033653+0000","last_peered":"2024-09-15T23:37:16.033653+0000","last_clean":"2024-09-15T23:37:16.033653+0000","last_became_active":"2024-09-15T23:37:16.031688+0000","last_became_peered":"2024-09-15T23:37:16.031688+0000","last_unstale":"2024-09-15T23:37:16.033653+0000","last_undegraded":"2024-09-15T23:37:16.033653+0000","last_fullsized":"2024-09-15T23:37:16.033653+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:36.654534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052263+0000","last_change":"2024-09-15T23:37:15.029001+0000","last_active":"2024-09-15T23:37:16.052263+0000","last_peered":"2024-09-15T23:37:16.052263+0000","last_clean":"2024-09-15T23:37:16.052263+0000","last_became_active":"2024-09-15T23:37:15.028669+0000","last_became_peered":"2024-09-15T23:37:15.028669+0000","last_unstale":"2024-09-15T23:37:16.052263+0000","last_undegraded":"2024-09-15T23:37:16.052263+0000","last_fullsized":"2024-09-15T23:37:16.052263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:32.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035709+0000","last_change":"2024-09-15T23:37:16.034125+0000","last_active":"2024-09-15T23:37:16.035709+0000","last_peered":"2024-09-15T23:37:16.035709+0000","last_clean":"2024-09-15T23:37:16.035709+0000","last_became_active":"2024-09-15T23:37:16.033801+0000","last_became_peered":"2024-09-15T23:37:16.033801+0000","last_unstale":"2024-09-15T23:37:16.035709+0000","last_undegraded":"2024-09-15T23:37:16.035709+0000","last_fullsized":"2024-09-15T23:37:16.035709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:04.589184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008354+0000","last_change":"2024-09-15T23:37:14.008354+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008354+0000","last_undegraded":"2024-09-15T23:37:14.008354+0000","last_fullsized":"2024-09-15T23:37:14.008354+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013255+0000","last_change":"2024-09-15T23:37:15.013255+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013255+0000","last_undegraded":"2024-09-15T23:37:15.013255+0000","last_fullsized":"2024-09-15T23:37:15.013255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022594+0000","last_change":"2024-09-15T23:37:15.026937+0000","last_active":"2024-09-15T23:37:16.022594+0000","last_peered":"2024-09-15T23:37:16.022594+0000","last_clean":"2024-09-15T23:37:16.022594+0000","last_became_active":"2024-09-15T23:37:15.026293+0000","last_became_peered":"2024-09-15T23:37:15.026293+0000","last_unstale":"2024-09-15T23:37:16.022594+0000","last_undegraded":"2024-09-15T23:37:16.022594+0000","last_fullsized":"2024-09-15T23:37:16.022594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:29.524303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008569+0000","last_change":"2024-09-15T23:37:14.008569+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008569+0000","last_undegraded":"2024-09-15T23:37:14.008569+0000","last_fullsized":"2024-09-15T23:37:14.008569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.045715+0000","last_change":"2024-09-15T23:37:16.044875+0000","last_active":"2024-09-15T23:37:16.045715+0000","last_peered":"2024-09-15T23:37:16.045715+0000","last_clean":"2024-09-15T23:37:16.045715+0000","last_became_active":"2024-09-15T23:37:16.044600+0000","last_became_peered":"2024-09-15T23:37:16.044600+0000","last_unstale":"2024-09-15T23:37:16.045715+0000","last_undegraded":"2024-09-15T23:37:16.045715+0000","last_fullsized":"2024-09-15T23:37:16.045715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:32.153327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020234+0000","last_change":"2024-09-15T23:37:15.025268+0000","last_active":"2024-09-15T23:37:16.020234+0000","last_peered":"2024-09-15T23:37:16.020234+0000","last_clean":"2024-09-15T23:37:16.020234+0000","last_became_active":"2024-09-15T23:37:15.024965+0000","last_became_peered":"2024-09-15T23:37:15.024965+0000","last_unstale":"2024-09-15T23:37:16.020234+0000","last_undegraded":"2024-09-15T23:37:16.020234+0000","last_fullsized":"2024-09-15T23:37:16.020234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:41.312641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.053070+0000","last_change":"2024-09-15T23:37:15.025534+0000","last_active":"2024-09-15T23:37:16.053070+0000","last_peered":"2024-09-15T23:37:16.053070+0000","last_clean":"2024-09-15T23:37:16.053070+0000","last_became_active":"2024-09-15T23:37:15.023762+0000","last_became_peered":"2024-09-15T23:37:15.023762+0000","last_unstale":"2024-09-15T23:37:16.053070+0000","last_undegraded":"2024-09-15T23:37:16.053070+0000","last_fullsized":"2024-09-15T23:37:16.053070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:11.328653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051853+0000","last_change":"2024-09-15T23:37:16.043056+0000","last_active":"2024-09-15T23:37:16.051853+0000","last_peered":"2024-09-15T23:37:16.051853+0000","last_clean":"2024-09-15T23:37:16.051853+0000","last_became_active":"2024-09-15T23:37:16.042639+0000","last_became_peered":"2024-09-15T23:37:16.042639+0000","last_unstale":"2024-09-15T23:37:16.051853+0000","last_undegraded":"2024-09-15T23:37:16.051853+0000","last_fullsized":"2024-09-15T23:37:16.051853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:51.445196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008657+0000","last_change":"2024-09-15T23:37:14.008657+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008657+0000","last_undegraded":"2024-09-15T23:37:14.008657+0000","last_fullsized":"2024-09-15T23:37:14.008657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.009242+0000","last_change":"2024-09-15T23:37:14.009242+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.009242+0000","last_undegraded":"2024-09-15T23:37:14.009242+0000","last_fullsized":"2024-09-15T23:37:14.009242+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015544+0000","last_change":"2024-09-15T23:37:15.015544+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015544+0000","last_undegraded":"2024-09-15T23:37:15.015544+0000","last_fullsized":"2024-09-15T23:37:15.015544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.023511+0000","last_change":"2024-09-15T23:37:15.021306+0000","last_active":"2024-09-15T23:37:15.023511+0000","last_peered":"2024-09-15T23:37:15.023511+0000","last_clean":"2024-09-15T23:37:15.023511+0000","last_became_active":"2024-09-15T23:37:15.020973+0000","last_became_peered":"2024-09-15T23:37:15.020973+0000","last_unstale":"2024-09-15T23:37:15.023511+0000","last_undegraded":"2024-09-15T23:37:15.023511+0000","last_fullsized":"2024-09-15T23:37:15.023511+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:09.094953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.034232+0000","last_change":"2024-09-15T23:37:16.028516+0000","last_active":"2024-09-15T23:37:16.034232+0000","last_peered":"2024-09-15T23:37:16.034232+0000","last_clean":"2024-09-15T23:37:16.034232+0000","last_became_active":"2024-09-15T23:37:16.028203+0000","last_became_peered":"2024-09-15T23:37:16.028203+0000","last_unstale":"2024-09-15T23:37:16.034232+0000","last_undegraded":"2024-09-15T23:37:16.034232+0000","last_fullsized":"2024-09-15T23:37:16.034232+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020739+0000","last_change":"2024-09-15T23:37:15.020887+0000","last_active":"2024-09-15T23:37:16.020739+0000","last_peered":"2024-09-15T23:37:16.020739+0000","last_clean":"2024-09-15T23:37:16.020739+0000","last_became_active":"2024-09-15T23:37:15.020611+0000","last_became_peered":"2024-09-15T23:37:15.020611+0000","last_unstale":"2024-09-15T23:37:16.020739+0000","last_undegraded":"2024-09-15T23:37:16.020739+0000","last_fullsized":"2024-09-15T23:37:16.020739+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:55.272773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013244+0000","last_change":"2024-09-15T23:37:15.013244+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013244+0000","last_undegraded":"2024-09-15T23:37:15.013244+0000","last_fullsized":"2024-09-15T23:37:15.013244+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.058124+0000","last_change":"2024-09-15T23:37:15.018587+0000","last_active":"2024-09-15T23:37:16.058124+0000","last_peered":"2024-09-15T23:37:16.058124+0000","last_clean":"2024-09-15T23:37:16.058124+0000","last_became_active":"2024-09-15T23:37:15.018296+0000","last_became_peered":"2024-09-15T23:37:15.018296+0000","last_unstale":"2024-09-15T23:37:16.058124+0000","last_undegraded":"2024-09-15T23:37:16.058124+0000","last_fullsized":"2024-09-15T23:37:16.058124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:01.784529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013616+0000","last_change":"2024-09-15T23:37:15.013616+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013616+0000","last_undegraded":"2024-09-15T23:37:15.013616+0000","last_fullsized":"2024-09-15T23:37:15.013616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052687+0000","last_change":"2024-09-15T23:37:15.025903+0000","last_active":"2024-09-15T23:37:16.052687+0000","last_peered":"2024-09-15T23:37:16.052687+0000","last_clean":"2024-09-15T23:37:16.052687+0000","last_became_active":"2024-09-15T23:37:15.025212+0000","last_became_peered":"2024-09-15T23:37:15.025212+0000","last_unstale":"2024-09-15T23:37:16.052687+0000","last_undegraded":"2024-09-15T23:37:16.052687+0000","last_fullsized":"2024-09-15T23:37:16.052687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:41.537186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047929+0000","last_change":"2024-09-15T23:37:16.042373+0000","last_active":"2024-09-15T23:37:16.047929+0000","last_peered":"2024-09-15T23:37:16.047929+0000","last_clean":"2024-09-15T23:37:16.047929+0000","last_became_active":"2024-09-15T23:37:16.040111+0000","last_became_peered":"2024-09-15T23:37:16.040111+0000","last_unstale":"2024-09-15T23:37:16.047929+0000","last_undegraded":"2024-09-15T23:37:16.047929+0000","last_fullsized":"2024-09-15T23:37:16.047929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:36:14.588537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052462+0000","last_change":"2024-09-15T23:37:15.024606+0000","last_active":"2024-09-15T23:37:16.052462+0000","last_peered":"2024-09-15T23:37:16.052462+0000","last_clean":"2024-09-15T23:37:16.052462+0000","last_became_active":"2024-09-15T23:37:15.024349+0000","last_became_peered":"2024-09-15T23:37:15.024349+0000","last_unstale":"2024-09-15T23:37:16.052462+0000","last_undegraded":"2024-09-15T23:37:16.052462+0000","last_fullsized":"2024-09-15T23:37:16.052462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:23.692350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047894+0000","last_change":"2024-09-15T23:37:16.041862+0000","last_active":"2024-09-15T23:37:16.047894+0000","last_peered":"2024-09-15T23:37:16.047894+0000","last_clean":"2024-09-15T23:37:16.047894+0000","last_became_active":"2024-09-15T23:37:16.041522+0000","last_became_peered":"2024-09-15T23:37:16.041522+0000","last_unstale":"2024-09-15T23:37:16.047894+0000","last_undegraded":"2024-09-15T23:37:16.047894+0000","last_fullsized":"2024-09-15T23:37:16.047894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:08.290613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.025129+0000","last_change":"2024-09-15T23:37:15.023432+0000","last_active":"2024-09-15T23:37:15.025129+0000","last_peered":"2024-09-15T23:37:15.025129+0000","last_clean":"2024-09-15T23:37:15.025129+0000","last_became_active":"2024-09-15T23:37:15.023142+0000","last_became_peered":"2024-09-15T23:37:15.023142+0000","last_unstale":"2024-09-15T23:37:15.025129+0000","last_undegraded":"2024-09-15T23:37:15.025129+0000","last_fullsized":"2024-09-15T23:37:15.025129+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:41.537186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.011492+0000","last_change":"2024-09-15T23:37:14.011492+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.011492+0000","last_undegraded":"2024-09-15T23:37:14.011492+0000","last_fullsized":"2024-09-15T23:37:14.011492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.025069+0000","last_change":"2024-09-15T23:37:16.023795+0000","last_active":"2024-09-15T23:37:16.025069+0000","last_peered":"2024-09-15T23:37:16.025069+0000","last_clean":"2024-09-15T23:37:16.025069+0000","last_became_active":"2024-09-15T23:37:16.023493+0000","last_became_peered":"2024-09-15T23:37:16.023493+0000","last_unstale":"2024-09-15T23:37:16.025069+0000","last_undegraded":"2024-09-15T23:37:16.025069+0000","last_fullsized":"2024-09-15T23:37:16.025069+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:19.252190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.011446+0000","last_change":"2024-09-15T23:37:14.011446+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.011446+0000","last_undegraded":"2024-09-15T23:37:14.011446+0000","last_fullsized":"2024-09-15T23:37:14.011446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.022413+0000","last_change":"2024-09-15T23:37:15.020340+0000","last_active":"2024-09-15T23:37:15.022413+0000","last_peered":"2024-09-15T23:37:15.022413+0000","last_clean":"2024-09-15T23:37:15.022413+0000","last_became_active":"2024-09-15T23:37:15.020031+0000","last_became_peered":"2024-09-15T23:37:15.020031+0000","last_unstale":"2024-09-15T23:37:15.022413+0000","last_undegraded":"2024-09-15T23:37:15.022413+0000","last_fullsized":"2024-09-15T23:37:15.022413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:50.440261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051420+0000","last_change":"2024-09-15T23:37:16.049420+0000","last_active":"2024-09-15T23:37:16.051420+0000","last_peered":"2024-09-15T23:37:16.051420+0000","last_clean":"2024-09-15T23:37:16.051420+0000","last_became_active":"2024-09-15T23:37:16.049125+0000","last_became_peered":"2024-09-15T23:37:16.049125+0000","last_unstale":"2024-09-15T23:37:16.051420+0000","last_undegraded":"2024-09-15T23:37:16.051420+0000","last_fullsized":"2024-09-15T23:37:16.051420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:56.834193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.024060+0000","last_change":"2024-09-15T23:37:15.030423+0000","last_active":"2024-09-15T23:37:16.024060+0000","last_peered":"2024-09-15T23:37:16.024060+0000","last_clean":"2024-09-15T23:37:16.024060+0000","last_became_active":"2024-09-15T23:37:15.030077+0000","last_became_peered":"2024-09-15T23:37:15.030077+0000","last_unstale":"2024-09-15T23:37:16.024060+0000","last_undegraded":"2024-09-15T23:37:16.024060+0000","last_fullsized":"2024-09-15T23:37:16.024060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.262938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.021265+0000","last_change":"2024-09-15T23:37:15.018265+0000","last_active":"2024-09-15T23:37:15.021265+0000","last_peered":"2024-09-15T23:37:15.021265+0000","last_clean":"2024-09-15T23:37:15.021265+0000","last_became_active":"2024-09-15T23:37:15.017961+0000","last_became_peered":"2024-09-15T23:37:15.017961+0000","last_unstale":"2024-09-15T23:37:15.021265+0000","last_undegraded":"2024-09-15T23:37:15.021265+0000","last_fullsized":"2024-09-15T23:37:15.021265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:36:13.583601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014174+0000","last_change":"2024-09-15T23:37:15.014174+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014174+0000","last_undegraded":"2024-09-15T23:37:15.014174+0000","last_fullsized":"2024-09-15T23:37:15.014174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052977+0000","last_change":"2024-09-15T23:37:15.021730+0000","last_active":"2024-09-15T23:37:16.052977+0000","last_peered":"2024-09-15T23:37:16.052977+0000","last_clean":"2024-09-15T23:37:16.052977+0000","last_became_active":"2024-09-15T23:37:15.021217+0000","last_became_peered":"2024-09-15T23:37:15.021217+0000","last_unstale":"2024-09-15T23:37:16.052977+0000","last_undegraded":"2024-09-15T23:37:16.052977+0000","last_fullsized":"2024-09-15T23:37:16.052977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:43:54.440674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052881+0000","last_change":"2024-09-15T23:37:16.050557+0000","last_active":"2024-09-15T23:37:16.052881+0000","last_peered":"2024-09-15T23:37:16.052881+0000","last_clean":"2024-09-15T23:37:16.052881+0000","last_became_active":"2024-09-15T23:37:16.050267+0000","last_became_peered":"2024-09-15T23:37:16.050267+0000","last_unstale":"2024-09-15T23:37:16.052881+0000","last_undegraded":"2024-09-15T23:37:16.052881+0000","last_fullsized":"2024-09-15T23:37:16.052881+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:01.512554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.030092+0000","last_change":"2024-09-15T23:37:15.029187+0000","last_active":"2024-09-15T23:37:15.030092+0000","last_peered":"2024-09-15T23:37:15.030092+0000","last_clean":"2024-09-15T23:37:15.030092+0000","last_became_active":"2024-09-15T23:37:15.028731+0000","last_became_peered":"2024-09-15T23:37:15.028731+0000","last_unstale":"2024-09-15T23:37:15.030092+0000","last_undegraded":"2024-09-15T23:37:15.030092+0000","last_fullsized":"2024-09-15T23:37:15.030092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.262938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.029676+0000","last_change":"2024-09-15T23:37:15.028765+0000","last_active":"2024-09-15T23:37:15.029676+0000","last_peered":"2024-09-15T23:37:15.029676+0000","last_clean":"2024-09-15T23:37:15.029676+0000","last_became_active":"2024-09-15T23:37:15.028426+0000","last_became_peered":"2024-09-15T23:37:15.028426+0000","last_unstale":"2024-09-15T23:37:15.029676+0000","last_undegraded":"2024-09-15T23:37:15.029676+0000","last_fullsized":"2024-09-15T23:37:15.029676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:32.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.012677+0000","last_change":"2024-09-15T23:37:15.012677+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.012677+0000","last_undegraded":"2024-09-15T23:37:15.012677+0000","last_fullsized":"2024-09-15T23:37:15.012677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008364+0000","last_change":"2024-09-15T23:37:14.008364+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008364+0000","last_undegraded":"2024-09-15T23:37:14.008364+0000","last_fullsized":"2024-09-15T23:37:14.008364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013767+0000","last_change":"2024-09-15T23:37:15.013767+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013767+0000","last_undegraded":"2024-09-15T23:37:15.013767+0000","last_fullsized":"2024-09-15T23:37:15.013767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.026104+0000","last_change":"2024-09-15T23:37:15.023867+0000","last_active":"2024-09-15T23:37:16.026104+0000","last_peered":"2024-09-15T23:37:16.026104+0000","last_clean":"2024-09-15T23:37:16.026104+0000","last_became_active":"2024-09-15T23:37:15.023470+0000","last_became_peered":"2024-09-15T23:37:15.023470+0000","last_unstale":"2024-09-15T23:37:16.026104+0000","last_undegraded":"2024-09-15T23:37:16.026104+0000","last_fullsized":"2024-09-15T23:37:16.026104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:13.335910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044234+0000","last_change":"2024-09-15T23:37:16.041472+0000","last_active":"2024-09-15T23:37:16.044234+0000","last_peered":"2024-09-15T23:37:16.044234+0000","last_clean":"2024-09-15T23:37:16.044234+0000","last_became_active":"2024-09-15T23:37:16.040586+0000","last_became_peered":"2024-09-15T23:37:16.040586+0000","last_unstale":"2024-09-15T23:37:16.044234+0000","last_undegraded":"2024-09-15T23:37:16.044234+0000","last_fullsized":"2024-09-15T23:37:16.044234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:01.946016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020562+0000","last_change":"2024-09-15T23:37:15.022433+0000","last_active":"2024-09-15T23:37:16.020562+0000","last_peered":"2024-09-15T23:37:16.020562+0000","last_clean":"2024-09-15T23:37:16.020562+0000","last_became_active":"2024-09-15T23:37:15.022150+0000","last_became_peered":"2024-09-15T23:37:15.022150+0000","last_unstale":"2024-09-15T23:37:16.020562+0000","last_undegraded":"2024-09-15T23:37:16.020562+0000","last_fullsized":"2024-09-15T23:37:16.020562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:06.425003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033731+0000","last_change":"2024-09-15T23:37:16.031076+0000","last_active":"2024-09-15T23:37:16.033731+0000","last_peered":"2024-09-15T23:37:16.033731+0000","last_clean":"2024-09-15T23:37:16.033731+0000","last_became_active":"2024-09-15T23:37:16.030718+0000","last_became_peered":"2024-09-15T23:37:16.030718+0000","last_unstale":"2024-09-15T23:37:16.033731+0000","last_undegraded":"2024-09-15T23:37:16.033731+0000","last_fullsized":"2024-09-15T23:37:16.033731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:31.411462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008468+0000","last_change":"2024-09-15T23:37:14.008468+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008468+0000","last_undegraded":"2024-09-15T23:37:14.008468+0000","last_fullsized":"2024-09-15T23:37:14.008468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035661+0000","last_change":"2024-09-15T23:37:16.034326+0000","last_active":"2024-09-15T23:37:16.035661+0000","last_peered":"2024-09-15T23:37:16.035661+0000","last_clean":"2024-09-15T23:37:16.035661+0000","last_became_active":"2024-09-15T23:37:16.033729+0000","last_became_peered":"2024-09-15T23:37:16.033729+0000","last_unstale":"2024-09-15T23:37:16.035661+0000","last_undegraded":"2024-09-15T23:37:16.035661+0000","last_fullsized":"2024-09-15T23:37:16.035661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:50.331476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.027426+0000","last_change":"2024-09-15T23:37:15.024501+0000","last_active":"2024-09-15T23:37:16.027426+0000","last_peered":"2024-09-15T23:37:16.027426+0000","last_clean":"2024-09-15T23:37:16.027426+0000","last_became_active":"2024-09-15T23:37:15.024195+0000","last_became_peered":"2024-09-15T23:37:15.024195+0000","last_unstale":"2024-09-15T23:37:16.027426+0000","last_undegraded":"2024-09-15T23:37:16.027426+0000","last_fullsized":"2024-09-15T23:37:16.027426+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:31.148391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022665+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:37:16.022665+0000","last_peered":"2024-09-15T23:37:16.022665+0000","last_clean":"2024-09-15T23:37:16.022665+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:16.022665+0000","last_undegraded":"2024-09-15T23:37:16.022665+0000","last_fullsized":"2024-09-15T23:37:16.022665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":185007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":446464,"data_stored":187201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1028096,"data_stored":775397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":438272,"data_stored":185099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":417792,"data_stored":177840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":417792,"data_stored":177840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":761541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":171173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:37:19.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:37:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.148+0000 7f0f6988d640 1 Processor -- start 2024-09-15T23:37:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.148+0000 7f0f6988d640 1 -- start start 2024-09-15T23:37:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.149+0000 7f0f6988d640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0f640d52e0 0x7f0f6406d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.149+0000 7f0f6988d640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f640d0a30 con 0x7f0f6415e840 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.149+0000 7f0f6988d640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f640cf560 con 0x7f0f6406dbb0 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.149+0000 7f0f6988d640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f0f640cfe90 con 0x7f0f640d52e0 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.149+0000 7f0f62ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0f6415e840 0x7f0f640d0320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51936/0 (socket says 172.21.15.7:51936) 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.149+0000 7f0f62ffd640 1 -- 172.21.15.7:0/2134312759 learned_addr learned my addr 172.21.15.7:0/2134312759 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 70786238 0 0) 0x7f0f640d0a30 con 0x7f0f6415e840 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f50003520 con 0x7f0f6415e840 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f627fc640 1 --2- 172.21.15.7:0/2134312759 >> v2:172.21.15.18:3300/0 conn(0x7f0f640d52e0 0x7f0f6406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3526904423 0 0) 0x7f0f50003520 con 0x7f0f6415e840 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 >> v2:172.21.15.18:3300/0 conn(0x7f0f640d52e0 msgr2=0x7f0f6406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 --2- 172.21.15.7:0/2134312759 >> v2:172.21.15.18:3300/0 conn(0x7f0f640d52e0 0x7f0f6406d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 >> v1:172.21.15.18:6789/0 conn(0x7f0f6406dbb0 legacy=0x7f0f640d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f640cf840 con 0x7f0f6415e840 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f0f540019f0 con 0x7f0f6415e840 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 >> v1:172.21.15.7:6789/0 conn(0x7f0f6415e840 legacy=0x7f0f640d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f50004800 0x7f0f50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f500051c0 0x7f0f5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640cfe90 con 0x7f0f500039b0 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5000c110 con 0x7f0f50004800 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5000c310 con 0x7f0f500051c0 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f62ffd640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f627fc640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f50004800 0x7f0f50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f637fe640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f500051c0 0x7f0f5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f62ffd640 1 -- 172.21.15.7:0/2134312759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f500051c0 msgr2=0x7f0f5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f62ffd640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f500051c0 0x7f0f5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f62ffd640 1 -- 172.21.15.7:0/2134312759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f50004800 msgr2=0x7f0f50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f62ffd640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f50004800 0x7f0f50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f62ffd640 1 -- 172.21.15.7:0/2134312759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f50003520 con 0x7f0f500039b0 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f637fe640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f500051c0 0x7f0f5000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.150+0000 7f0f637fe640 1 -- 172.21.15.7:0/2134312759 reap_dead start 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.151+0000 7f0f62ffd640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f50003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400a860 tx=0x7f0f50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b98890c683ffb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.151+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400a480 con 0x7f0f500039b0 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.151+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5402f070 con 0x7f0f500039b0 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.151+0000 7f0f61ffb640 1 -- 172.21.15.7:0/2134312759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5402bd40 con 0x7f0f500039b0 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.152+0000 7f0f6988d640 1 -- 172.21.15.7:0/2134312759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 msgr2=0x7f0f50003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.152+0000 7f0f6988d640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f50003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400a860 tx=0x7f0f50005830 comp rx=0 tx=0).stop 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.152+0000 7f0f6988d640 1 -- 172.21.15.7:0/2134312759 shutdown_connections 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.152+0000 7f0f6988d640 1 --2- 172.21.15.7:0/2134312759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f50003dd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.152+0000 7f0f6988d640 1 -- 172.21.15.7:0/2134312759 >> 172.21.15.7:0/2134312759 conn(0x7f0f6408bfd0 msgr2=0x7f0f640d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.152+0000 7f0f6988d640 1 -- 172.21.15.7:0/2134312759 shutdown_connections 2024-09-15T23:37:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.152+0000 7f0f6988d640 1 -- 172.21.15.7:0/2134312759 wait complete. 2024-09-15T23:37:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.153+0000 7f0f6988d640 1 Processor -- start 2024-09-15T23:37:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.153+0000 7f0f6988d640 1 -- start start 2024-09-15T23:37:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f6988d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f6414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f6988d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f64157420 0x7f0f641578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f6988d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f640c80f0 0x7f0f640c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f6988d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6415ec20 con 0x7f0f500039b0 2024-09-15T23:37:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f6988d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6406e030 con 0x7f0f640c80f0 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f6988d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f0f640b6530 con 0x7f0f64157420 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f62ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f6414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f62ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f6414eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54506/0 (socket says 172.21.15.7:54506) 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f62ffd640 1 -- 172.21.15.7:0/3425419969 learned_addr learned my addr 172.21.15.7:0/3425419969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f627fc640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f64157420 0x7f0f641578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f637fe640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f640c80f0 0x7f0f640c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f62ffd640 1 -- 172.21.15.7:0/3425419969 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f64157420 msgr2=0x7f0f641578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f62ffd640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f64157420 0x7f0f641578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f62ffd640 1 -- 172.21.15.7:0/3425419969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f640c80f0 msgr2=0x7f0f640c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f62ffd640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f640c80f0 0x7f0f640c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.154+0000 7f0f62ffd640 1 -- 172.21.15.7:0/3425419969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f640b5640 con 0x7f0f500039b0 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.155+0000 7f0f62ffd640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f6414eb00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400a860 tx=0x7f0f640cf730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.155+0000 7f0f5bfff640 1 -- 172.21.15.7:0/3425419969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54044070 con 0x7f0f500039b0 2024-09-15T23:37:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.155+0000 7f0f5bfff640 1 -- 172.21.15.7:0/3425419969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5402be40 con 0x7f0f500039b0 2024-09-15T23:37:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.155+0000 7f0f5bfff640 1 -- 172.21.15.7:0/3425419969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400a480 con 0x7f0f500039b0 2024-09-15T23:37:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.155+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f640b5920 con 0x7f0f500039b0 2024-09-15T23:37:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.155+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f640b5ee0 con 0x7f0f500039b0 2024-09-15T23:37:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.156+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6406d670 con 0x7f0f500039b0 2024-09-15T23:37:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.156+0000 7f0f5bfff640 1 -- 172.21.15.7:0/3425419969 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0f54047070 con 0x7f0f500039b0 2024-09-15T23:37:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.157+0000 7f0f5bfff640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f0f4406c2b0 0x7f0f4406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.157+0000 7f0f5bfff640 1 -- 172.21.15.7:0/3425419969 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0f540ab220 con 0x7f0f500039b0 2024-09-15T23:37:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.157+0000 7f0f627fc640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f0f4406c2b0 0x7f0f4406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.157+0000 7f0f627fc640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f0f4406c2b0 0x7f0f4406e770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0f64157e70 tx=0x7f0f4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.161+0000 7f0f5bfff640 1 -- 172.21.15.7:0/3425419969 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0f540799b0 con 0x7f0f500039b0 2024-09-15T23:37:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.335+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f640d1cb0 con 0x7f0f4406c2b0 2024-09-15T23:37:19.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.341+0000 7f0f5bfff640 1 -- 172.21.15.7:0/3425419969 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247699 (secure 0 0 0) 0x7f0f640d1cb0 con 0x7f0f4406c2b0 2024-09-15T23:37:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f0f4406c2b0 msgr2=0x7f0f4406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f0f4406c2b0 0x7f0f4406e770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0f64157e70 tx=0x7f0f4c008040 comp rx=0 tx=0).stop 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 msgr2=0x7f0f6414eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f6414eb00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400a860 tx=0x7f0f640cf730 comp rx=0 tx=0).stop 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 shutdown_connections 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f640c80f0 0x7f0f640c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f0f4406c2b0 0x7f0f4406e770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0f64157420 0x7f0f641578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 --2- 172.21.15.7:0/3425419969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500039b0 0x7f0f6414eb00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 >> 172.21.15.7:0/3425419969 conn(0x7f0f6408bfd0 msgr2=0x7f0f640b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 shutdown_connections 2024-09-15T23:37:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.344+0000 7f0f6988d640 1 -- 172.21.15.7:0/3425419969 wait complete. 2024-09-15T23:37:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-15T23:37:17.823024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254004,"kb_used_data":4476,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720716,"statfs":{"total":773094113280,"available":772834013184,"internally_reserved":0,"allocated":4583424,"data_stored":2621098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000643"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014716+0000","last_change":"2024-09-15T23:37:15.014716+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014716+0000","last_undegraded":"2024-09-15T23:37:15.014716+0000","last_fullsized":"2024-09-15T23:37:15.014716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013817+0000","last_change":"2024-09-15T23:37:15.013817+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013817+0000","last_undegraded":"2024-09-15T23:37:15.013817+0000","last_fullsized":"2024-09-15T23:37:15.013817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015854+0000","last_change":"2024-09-15T23:37:15.015854+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015854+0000","last_undegraded":"2024-09-15T23:37:15.015854+0000","last_fullsized":"2024-09-15T23:37:15.015854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048204+0000","last_change":"2024-09-15T23:37:16.039624+0000","last_active":"2024-09-15T23:37:16.048204+0000","last_peered":"2024-09-15T23:37:16.048204+0000","last_clean":"2024-09-15T23:37:16.048204+0000","last_became_active":"2024-09-15T23:37:16.038879+0000","last_became_peered":"2024-09-15T23:37:16.038879+0000","last_unstale":"2024-09-15T23:37:16.048204+0000","last_undegraded":"2024-09-15T23:37:16.048204+0000","last_fullsized":"2024-09-15T23:37:16.048204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:51.545065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013261+0000","last_change":"2024-09-15T23:37:15.013261+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013261+0000","last_undegraded":"2024-09-15T23:37:15.013261+0000","last_fullsized":"2024-09-15T23:37:15.013261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051987+0000","last_change":"2024-09-15T23:37:16.043510+0000","last_active":"2024-09-15T23:37:16.051987+0000","last_peered":"2024-09-15T23:37:16.051987+0000","last_clean":"2024-09-15T23:37:16.051987+0000","last_became_active":"2024-09-15T23:37:16.043208+0000","last_became_peered":"2024-09-15T23:37:16.043208+0000","last_unstale":"2024-09-15T23:37:16.051987+0000","last_undegraded":"2024-09-15T23:37:16.051987+0000","last_fullsized":"2024-09-15T23:37:16.051987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:10.099889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052859+0000","last_change":"2024-09-15T23:37:16.050348+0000","last_active":"2024-09-15T23:37:16.052859+0000","last_peered":"2024-09-15T23:37:16.052859+0000","last_clean":"2024-09-15T23:37:16.052859+0000","last_became_active":"2024-09-15T23:37:16.049871+0000","last_became_peered":"2024-09-15T23:37:16.049871+0000","last_unstale":"2024-09-15T23:37:16.052859+0000","last_undegraded":"2024-09-15T23:37:16.052859+0000","last_fullsized":"2024-09-15T23:37:16.052859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:23.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015421+0000","last_change":"2024-09-15T23:37:15.015421+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015421+0000","last_undegraded":"2024-09-15T23:37:15.015421+0000","last_fullsized":"2024-09-15T23:37:15.015421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047271+0000","last_change":"2024-09-15T23:37:16.037928+0000","last_active":"2024-09-15T23:37:16.047271+0000","last_peered":"2024-09-15T23:37:16.047271+0000","last_clean":"2024-09-15T23:37:16.047271+0000","last_became_active":"2024-09-15T23:37:16.037645+0000","last_became_peered":"2024-09-15T23:37:16.037645+0000","last_unstale":"2024-09-15T23:37:16.047271+0000","last_undegraded":"2024-09-15T23:37:16.047271+0000","last_fullsized":"2024-09-15T23:37:16.047271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:29.660110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048427+0000","last_change":"2024-09-15T23:37:16.046081+0000","last_active":"2024-09-15T23:37:16.048427+0000","last_peered":"2024-09-15T23:37:16.048427+0000","last_clean":"2024-09-15T23:37:16.048427+0000","last_became_active":"2024-09-15T23:37:16.045733+0000","last_became_peered":"2024-09-15T23:37:16.045733+0000","last_unstale":"2024-09-15T23:37:16.048427+0000","last_undegraded":"2024-09-15T23:37:16.048427+0000","last_fullsized":"2024-09-15T23:37:16.048427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036156+0000","last_change":"2024-09-15T23:37:16.024098+0000","last_active":"2024-09-15T23:37:16.036156+0000","last_peered":"2024-09-15T23:37:16.036156+0000","last_clean":"2024-09-15T23:37:16.036156+0000","last_became_active":"2024-09-15T23:37:16.023753+0000","last_became_peered":"2024-09-15T23:37:16.023753+0000","last_unstale":"2024-09-15T23:37:16.036156+0000","last_undegraded":"2024-09-15T23:37:16.036156+0000","last_fullsized":"2024-09-15T23:37:16.036156+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:34.586227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052091+0000","last_change":"2024-09-15T23:37:16.048043+0000","last_active":"2024-09-15T23:37:16.052091+0000","last_peered":"2024-09-15T23:37:16.052091+0000","last_clean":"2024-09-15T23:37:16.052091+0000","last_became_active":"2024-09-15T23:37:16.047587+0000","last_became_peered":"2024-09-15T23:37:16.047587+0000","last_unstale":"2024-09-15T23:37:16.052091+0000","last_undegraded":"2024-09-15T23:37:16.052091+0000","last_fullsized":"2024-09-15T23:37:16.052091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:57.383248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035550+0000","last_change":"2024-09-15T23:37:16.034026+0000","last_active":"2024-09-15T23:37:16.035550+0000","last_peered":"2024-09-15T23:37:16.035550+0000","last_clean":"2024-09-15T23:37:16.035550+0000","last_became_active":"2024-09-15T23:37:16.033385+0000","last_became_peered":"2024-09-15T23:37:16.033385+0000","last_unstale":"2024-09-15T23:37:16.035550+0000","last_undegraded":"2024-09-15T23:37:16.035550+0000","last_fullsized":"2024-09-15T23:37:16.035550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:33.923553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035623+0000","last_change":"2024-09-15T23:37:16.034152+0000","last_active":"2024-09-15T23:37:16.035623+0000","last_peered":"2024-09-15T23:37:16.035623+0000","last_clean":"2024-09-15T23:37:16.035623+0000","last_became_active":"2024-09-15T23:37:16.029607+0000","last_became_peered":"2024-09-15T23:37:16.029607+0000","last_unstale":"2024-09-15T23:37:16.035623+0000","last_undegraded":"2024-09-15T23:37:16.035623+0000","last_fullsized":"2024-09-15T23:37:16.035623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:55.516820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042577+0000","last_change":"2024-09-15T23:37:16.037756+0000","last_active":"2024-09-15T23:37:16.042577+0000","last_peered":"2024-09-15T23:37:16.042577+0000","last_clean":"2024-09-15T23:37:16.042577+0000","last_became_active":"2024-09-15T23:37:16.037309+0000","last_became_peered":"2024-09-15T23:37:16.037309+0000","last_unstale":"2024-09-15T23:37:16.042577+0000","last_undegraded":"2024-09-15T23:37:16.042577+0000","last_fullsized":"2024-09-15T23:37:16.042577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:45.986835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042641+0000","last_change":"2024-09-15T23:37:16.037039+0000","last_active":"2024-09-15T23:37:16.042641+0000","last_peered":"2024-09-15T23:37:16.042641+0000","last_clean":"2024-09-15T23:37:16.042641+0000","last_became_active":"2024-09-15T23:37:16.036774+0000","last_became_peered":"2024-09-15T23:37:16.036774+0000","last_unstale":"2024-09-15T23:37:16.042641+0000","last_undegraded":"2024-09-15T23:37:16.042641+0000","last_fullsized":"2024-09-15T23:37:16.042641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:30.804394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.040576+0000","last_change":"2024-09-15T23:37:16.034353+0000","last_active":"2024-09-15T23:37:16.040576+0000","last_peered":"2024-09-15T23:37:16.040576+0000","last_clean":"2024-09-15T23:37:16.040576+0000","last_became_active":"2024-09-15T23:37:16.033979+0000","last_became_peered":"2024-09-15T23:37:16.033979+0000","last_unstale":"2024-09-15T23:37:16.040576+0000","last_undegraded":"2024-09-15T23:37:16.040576+0000","last_fullsized":"2024-09-15T23:37:16.040576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:05.128368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036134+0000","last_change":"2024-09-15T23:37:16.027056+0000","last_active":"2024-09-15T23:37:16.036134+0000","last_peered":"2024-09-15T23:37:16.036134+0000","last_clean":"2024-09-15T23:37:16.036134+0000","last_became_active":"2024-09-15T23:37:16.026780+0000","last_became_peered":"2024-09-15T23:37:16.026780+0000","last_unstale":"2024-09-15T23:37:16.036134+0000","last_undegraded":"2024-09-15T23:37:16.036134+0000","last_fullsized":"2024-09-15T23:37:16.036134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:43.279890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044498+0000","last_change":"2024-09-15T23:37:16.042298+0000","last_active":"2024-09-15T23:37:16.044498+0000","last_peered":"2024-09-15T23:37:16.044498+0000","last_clean":"2024-09-15T23:37:16.044498+0000","last_became_active":"2024-09-15T23:37:16.041961+0000","last_became_peered":"2024-09-15T23:37:16.041961+0000","last_unstale":"2024-09-15T23:37:16.044498+0000","last_undegraded":"2024-09-15T23:37:16.044498+0000","last_fullsized":"2024-09-15T23:37:16.044498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:14.340846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033699+0000","last_change":"2024-09-15T23:37:16.031292+0000","last_active":"2024-09-15T23:37:16.033699+0000","last_peered":"2024-09-15T23:37:16.033699+0000","last_clean":"2024-09-15T23:37:16.033699+0000","last_became_active":"2024-09-15T23:37:16.030975+0000","last_became_peered":"2024-09-15T23:37:16.030975+0000","last_unstale":"2024-09-15T23:37:16.033699+0000","last_undegraded":"2024-09-15T23:37:16.033699+0000","last_fullsized":"2024-09-15T23:37:16.033699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:12.360351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048310+0000","last_change":"2024-09-15T23:37:16.039927+0000","last_active":"2024-09-15T23:37:16.048310+0000","last_peered":"2024-09-15T23:37:16.048310+0000","last_clean":"2024-09-15T23:37:16.048310+0000","last_became_active":"2024-09-15T23:37:16.039622+0000","last_became_peered":"2024-09-15T23:37:16.039622+0000","last_unstale":"2024-09-15T23:37:16.048310+0000","last_undegraded":"2024-09-15T23:37:16.048310+0000","last_fullsized":"2024-09-15T23:37:16.048310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:54.978065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035781+0000","last_change":"2024-09-15T23:37:16.032318+0000","last_active":"2024-09-15T23:37:16.035781+0000","last_peered":"2024-09-15T23:37:16.035781+0000","last_clean":"2024-09-15T23:37:16.035781+0000","last_became_active":"2024-09-15T23:37:16.028787+0000","last_became_peered":"2024-09-15T23:37:16.028787+0000","last_unstale":"2024-09-15T23:37:16.035781+0000","last_undegraded":"2024-09-15T23:37:16.035781+0000","last_fullsized":"2024-09-15T23:37:16.035781+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:31.065835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035685+0000","last_change":"2024-09-15T23:37:16.028365+0000","last_active":"2024-09-15T23:37:16.035685+0000","last_peered":"2024-09-15T23:37:16.035685+0000","last_clean":"2024-09-15T23:37:16.035685+0000","last_became_active":"2024-09-15T23:37:16.028051+0000","last_became_peered":"2024-09-15T23:37:16.028051+0000","last_unstale":"2024-09-15T23:37:16.035685+0000","last_undegraded":"2024-09-15T23:37:16.035685+0000","last_fullsized":"2024-09-15T23:37:16.035685+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:47.439397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048225+0000","last_change":"2024-09-15T23:37:16.039870+0000","last_active":"2024-09-15T23:37:16.048225+0000","last_peered":"2024-09-15T23:37:16.048225+0000","last_clean":"2024-09-15T23:37:16.048225+0000","last_became_active":"2024-09-15T23:37:16.039295+0000","last_became_peered":"2024-09-15T23:37:16.039295+0000","last_unstale":"2024-09-15T23:37:16.048225+0000","last_undegraded":"2024-09-15T23:37:16.048225+0000","last_fullsized":"2024-09-15T23:37:16.048225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:16.959191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046178+0000","last_change":"2024-09-15T23:37:16.044114+0000","last_active":"2024-09-15T23:37:16.046178+0000","last_peered":"2024-09-15T23:37:16.046178+0000","last_clean":"2024-09-15T23:37:16.046178+0000","last_became_active":"2024-09-15T23:37:16.042005+0000","last_became_peered":"2024-09-15T23:37:16.042005+0000","last_unstale":"2024-09-15T23:37:16.046178+0000","last_undegraded":"2024-09-15T23:37:16.046178+0000","last_fullsized":"2024-09-15T23:37:16.046178+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:40.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056140+0000","last_change":"2024-09-15T23:37:15.019375+0000","last_active":"2024-09-15T23:37:16.056140+0000","last_peered":"2024-09-15T23:37:16.056140+0000","last_clean":"2024-09-15T23:37:16.056140+0000","last_became_active":"2024-09-15T23:37:15.018977+0000","last_became_peered":"2024-09-15T23:37:15.018977+0000","last_unstale":"2024-09-15T23:37:16.056140+0000","last_undegraded":"2024-09-15T23:37:16.056140+0000","last_fullsized":"2024-09-15T23:37:16.056140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:02.654673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013153+0000","last_change":"2024-09-15T23:37:15.013153+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013153+0000","last_undegraded":"2024-09-15T23:37:15.013153+0000","last_fullsized":"2024-09-15T23:37:15.013153+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.027944+0000","last_change":"2024-09-15T23:37:15.024812+0000","last_active":"2024-09-15T23:37:15.027944+0000","last_peered":"2024-09-15T23:37:15.027944+0000","last_clean":"2024-09-15T23:37:15.027944+0000","last_became_active":"2024-09-15T23:37:15.024533+0000","last_became_peered":"2024-09-15T23:37:15.024533+0000","last_unstale":"2024-09-15T23:37:15.027944+0000","last_undegraded":"2024-09-15T23:37:15.027944+0000","last_fullsized":"2024-09-15T23:37:15.027944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:56.378313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046817+0000","last_change":"2024-09-15T23:37:16.043209+0000","last_active":"2024-09-15T23:37:16.046817+0000","last_peered":"2024-09-15T23:37:16.046817+0000","last_clean":"2024-09-15T23:37:16.046817+0000","last_became_active":"2024-09-15T23:37:16.042016+0000","last_became_peered":"2024-09-15T23:37:16.042016+0000","last_unstale":"2024-09-15T23:37:16.046817+0000","last_undegraded":"2024-09-15T23:37:16.046817+0000","last_fullsized":"2024-09-15T23:37:16.046817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:17.167815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008364+0000","last_change":"2024-09-15T23:37:14.008364+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008364+0000","last_undegraded":"2024-09-15T23:37:14.008364+0000","last_fullsized":"2024-09-15T23:37:14.008364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014181+0000","last_change":"2024-09-15T23:37:15.014181+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014181+0000","last_undegraded":"2024-09-15T23:37:15.014181+0000","last_fullsized":"2024-09-15T23:37:15.014181+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056405+0000","last_change":"2024-09-15T23:37:15.023109+0000","last_active":"2024-09-15T23:37:16.056405+0000","last_peered":"2024-09-15T23:37:16.056405+0000","last_clean":"2024-09-15T23:37:16.056405+0000","last_became_active":"2024-09-15T23:37:15.022494+0000","last_became_peered":"2024-09-15T23:37:15.022494+0000","last_unstale":"2024-09-15T23:37:16.056405+0000","last_undegraded":"2024-09-15T23:37:16.056405+0000","last_fullsized":"2024-09-15T23:37:16.056405+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:29.332641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.055917+0000","last_change":"2024-09-15T23:37:15.025209+0000","last_active":"2024-09-15T23:37:16.055917+0000","last_peered":"2024-09-15T23:37:16.055917+0000","last_clean":"2024-09-15T23:37:16.055917+0000","last_became_active":"2024-09-15T23:37:15.022678+0000","last_became_peered":"2024-09-15T23:37:15.022678+0000","last_unstale":"2024-09-15T23:37:16.055917+0000","last_undegraded":"2024-09-15T23:37:16.055917+0000","last_fullsized":"2024-09-15T23:37:16.055917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:01.000785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033653+0000","last_change":"2024-09-15T23:37:16.031990+0000","last_active":"2024-09-15T23:37:16.033653+0000","last_peered":"2024-09-15T23:37:16.033653+0000","last_clean":"2024-09-15T23:37:16.033653+0000","last_became_active":"2024-09-15T23:37:16.031688+0000","last_became_peered":"2024-09-15T23:37:16.031688+0000","last_unstale":"2024-09-15T23:37:16.033653+0000","last_undegraded":"2024-09-15T23:37:16.033653+0000","last_fullsized":"2024-09-15T23:37:16.033653+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:36.654534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052263+0000","last_change":"2024-09-15T23:37:15.029001+0000","last_active":"2024-09-15T23:37:16.052263+0000","last_peered":"2024-09-15T23:37:16.052263+0000","last_clean":"2024-09-15T23:37:16.052263+0000","last_became_active":"2024-09-15T23:37:15.028669+0000","last_became_peered":"2024-09-15T23:37:15.028669+0000","last_unstale":"2024-09-15T23:37:16.052263+0000","last_undegraded":"2024-09-15T23:37:16.052263+0000","last_fullsized":"2024-09-15T23:37:16.052263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:32.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035709+0000","last_change":"2024-09-15T23:37:16.034125+0000","last_active":"2024-09-15T23:37:16.035709+0000","last_peered":"2024-09-15T23:37:16.035709+0000","last_clean":"2024-09-15T23:37:16.035709+0000","last_became_active":"2024-09-15T23:37:16.033801+0000","last_became_peered":"2024-09-15T23:37:16.033801+0000","last_unstale":"2024-09-15T23:37:16.035709+0000","last_undegraded":"2024-09-15T23:37:16.035709+0000","last_fullsized":"2024-09-15T23:37:16.035709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:04.589184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008354+0000","last_change":"2024-09-15T23:37:14.008354+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008354+0000","last_undegraded":"2024-09-15T23:37:14.008354+0000","last_fullsized":"2024-09-15T23:37:14.008354+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013255+0000","last_change":"2024-09-15T23:37:15.013255+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013255+0000","last_undegraded":"2024-09-15T23:37:15.013255+0000","last_fullsized":"2024-09-15T23:37:15.013255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022594+0000","last_change":"2024-09-15T23:37:15.026937+0000","last_active":"2024-09-15T23:37:16.022594+0000","last_peered":"2024-09-15T23:37:16.022594+0000","last_clean":"2024-09-15T23:37:16.022594+0000","last_became_active":"2024-09-15T23:37:15.026293+0000","last_became_peered":"2024-09-15T23:37:15.026293+0000","last_unstale":"2024-09-15T23:37:16.022594+0000","last_undegraded":"2024-09-15T23:37:16.022594+0000","last_fullsized":"2024-09-15T23:37:16.022594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:29.524303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008569+0000","last_change":"2024-09-15T23:37:14.008569+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008569+0000","last_undegraded":"2024-09-15T23:37:14.008569+0000","last_fullsized":"2024-09-15T23:37:14.008569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.045715+0000","last_change":"2024-09-15T23:37:16.044875+0000","last_active":"2024-09-15T23:37:16.045715+0000","last_peered":"2024-09-15T23:37:16.045715+0000","last_clean":"2024-09-15T23:37:16.045715+0000","last_became_active":"2024-09-15T23:37:16.044600+0000","last_became_peered":"2024-09-15T23:37:16.044600+0000","last_unstale":"2024-09-15T23:37:16.045715+0000","last_undegraded":"2024-09-15T23:37:16.045715+0000","last_fullsized":"2024-09-15T23:37:16.045715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:32.153327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020234+0000","last_change":"2024-09-15T23:37:15.025268+0000","last_active":"2024-09-15T23:37:16.020234+0000","last_peered":"2024-09-15T23:37:16.020234+0000","last_clean":"2024-09-15T23:37:16.020234+0000","last_became_active":"2024-09-15T23:37:15.024965+0000","last_became_peered":"2024-09-15T23:37:15.024965+0000","last_unstale":"2024-09-15T23:37:16.020234+0000","last_undegraded":"2024-09-15T23:37:16.020234+0000","last_fullsized":"2024-09-15T23:37:16.020234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:41.312641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.053070+0000","last_change":"2024-09-15T23:37:15.025534+0000","last_active":"2024-09-15T23:37:16.053070+0000","last_peered":"2024-09-15T23:37:16.053070+0000","last_clean":"2024-09-15T23:37:16.053070+0000","last_became_active":"2024-09-15T23:37:15.023762+0000","last_became_peered":"2024-09-15T23:37:15.023762+0000","last_unstale":"2024-09-15T23:37:16.053070+0000","last_undegraded":"2024-09-15T23:37:16.053070+0000","last_fullsized":"2024-09-15T23:37:16.053070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:11.328653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051853+0000","last_change":"2024-09-15T23:37:16.043056+0000","last_active":"2024-09-15T23:37:16.051853+0000","last_peered":"2024-09-15T23:37:16.051853+0000","last_clean":"2024-09-15T23:37:16.051853+0000","last_became_active":"2024-09-15T23:37:16.042639+0000","last_became_peered":"2024-09-15T23:37:16.042639+0000","last_unstale":"2024-09-15T23:37:16.051853+0000","last_undegraded":"2024-09-15T23:37:16.051853+0000","last_fullsized":"2024-09-15T23:37:16.051853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:51.445196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008657+0000","last_change":"2024-09-15T23:37:14.008657+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008657+0000","last_undegraded":"2024-09-15T23:37:14.008657+0000","last_fullsized":"2024-09-15T23:37:14.008657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.009242+0000","last_change":"2024-09-15T23:37:14.009242+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.009242+0000","last_undegraded":"2024-09-15T23:37:14.009242+0000","last_fullsized":"2024-09-15T23:37:14.009242+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015544+0000","last_change":"2024-09-15T23:37:15.015544+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015544+0000","last_undegraded":"2024-09-15T23:37:15.015544+0000","last_fullsized":"2024-09-15T23:37:15.015544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.023511+0000","last_change":"2024-09-15T23:37:15.021306+0000","last_active":"2024-09-15T23:37:15.023511+0000","last_peered":"2024-09-15T23:37:15.023511+0000","last_clean":"2024-09-15T23:37:15.023511+0000","last_became_active":"2024-09-15T23:37:15.020973+0000","last_became_peered":"2024-09-15T23:37:15.020973+0000","last_unstale":"2024-09-15T23:37:15.023511+0000","last_undegraded":"2024-09-15T23:37:15.023511+0000","last_fullsized":"2024-09-15T23:37:15.023511+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:09.094953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.034232+0000","last_change":"2024-09-15T23:37:16.028516+0000","last_active":"2024-09-15T23:37:16.034232+0000","last_peered":"2024-09-15T23:37:16.034232+0000","last_clean":"2024-09-15T23:37:16.034232+0000","last_became_active":"2024-09-15T23:37:16.028203+0000","last_became_peered":"2024-09-15T23:37:16.028203+0000","last_unstale":"2024-09-15T23:37:16.034232+0000","last_undegraded":"2024-09-15T23:37:16.034232+0000","last_fullsized":"2024-09-15T23:37:16.034232+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020739+0000","last_change":"2024-09-15T23:37:15.020887+0000","last_active":"2024-09-15T23:37:16.020739+0000","last_peered":"2024-09-15T23:37:16.020739+0000","last_clean":"2024-09-15T23:37:16.020739+0000","last_became_active":"2024-09-15T23:37:15.020611+0000","last_became_peered":"2024-09-15T23:37:15.020611+0000","last_unstale":"2024-09-15T23:37:16.020739+0000","last_undegraded":"2024-09-15T23:37:16.020739+0000","last_fullsized":"2024-09-15T23:37:16.020739+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:55.272773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013244+0000","last_change":"2024-09-15T23:37:15.013244+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013244+0000","last_undegraded":"2024-09-15T23:37:15.013244+0000","last_fullsized":"2024-09-15T23:37:15.013244+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.058124+0000","last_change":"2024-09-15T23:37:15.018587+0000","last_active":"2024-09-15T23:37:16.058124+0000","last_peered":"2024-09-15T23:37:16.058124+0000","last_clean":"2024-09-15T23:37:16.058124+0000","last_became_active":"2024-09-15T23:37:15.018296+0000","last_became_peered":"2024-09-15T23:37:15.018296+0000","last_unstale":"2024-09-15T23:37:16.058124+0000","last_undegraded":"2024-09-15T23:37:16.058124+0000","last_fullsized":"2024-09-15T23:37:16.058124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:01.784529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013616+0000","last_change":"2024-09-15T23:37:15.013616+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013616+0000","last_undegraded":"2024-09-15T23:37:15.013616+0000","last_fullsized":"2024-09-15T23:37:15.013616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052687+0000","last_change":"2024-09-15T23:37:15.025903+0000","last_active":"2024-09-15T23:37:16.052687+0000","last_peered":"2024-09-15T23:37:16.052687+0000","last_clean":"2024-09-15T23:37:16.052687+0000","last_became_active":"2024-09-15T23:37:15.025212+0000","last_became_peered":"2024-09-15T23:37:15.025212+0000","last_unstale":"2024-09-15T23:37:16.052687+0000","last_undegraded":"2024-09-15T23:37:16.052687+0000","last_fullsized":"2024-09-15T23:37:16.052687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:41.537186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047929+0000","last_change":"2024-09-15T23:37:16.042373+0000","last_active":"2024-09-15T23:37:16.047929+0000","last_peered":"2024-09-15T23:37:16.047929+0000","last_clean":"2024-09-15T23:37:16.047929+0000","last_became_active":"2024-09-15T23:37:16.040111+0000","last_became_peered":"2024-09-15T23:37:16.040111+0000","last_unstale":"2024-09-15T23:37:16.047929+0000","last_undegraded":"2024-09-15T23:37:16.047929+0000","last_fullsized":"2024-09-15T23:37:16.047929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:36:14.588537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052462+0000","last_change":"2024-09-15T23:37:15.024606+0000","last_active":"2024-09-15T23:37:16.052462+0000","last_peered":"2024-09-15T23:37:16.052462+0000","last_clean":"2024-09-15T23:37:16.052462+0000","last_became_active":"2024-09-15T23:37:15.024349+0000","last_became_peered":"2024-09-15T23:37:15.024349+0000","last_unstale":"2024-09-15T23:37:16.052462+0000","last_undegraded":"2024-09-15T23:37:16.052462+0000","last_fullsized":"2024-09-15T23:37:16.052462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:23.692350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047894+0000","last_change":"2024-09-15T23:37:16.041862+0000","last_active":"2024-09-15T23:37:16.047894+0000","last_peered":"2024-09-15T23:37:16.047894+0000","last_clean":"2024-09-15T23:37:16.047894+0000","last_became_active":"2024-09-15T23:37:16.041522+0000","last_became_peered":"2024-09-15T23:37:16.041522+0000","last_unstale":"2024-09-15T23:37:16.047894+0000","last_undegraded":"2024-09-15T23:37:16.047894+0000","last_fullsized":"2024-09-15T23:37:16.047894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:08.290613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.025129+0000","last_change":"2024-09-15T23:37:15.023432+0000","last_active":"2024-09-15T23:37:15.025129+0000","last_peered":"2024-09-15T23:37:15.025129+0000","last_clean":"2024-09-15T23:37:15.025129+0000","last_became_active":"2024-09-15T23:37:15.023142+0000","last_became_peered":"2024-09-15T23:37:15.023142+0000","last_unstale":"2024-09-15T23:37:15.025129+0000","last_undegraded":"2024-09-15T23:37:15.025129+0000","last_fullsized":"2024-09-15T23:37:15.025129+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:41.537186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.011492+0000","last_change":"2024-09-15T23:37:14.011492+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.011492+0000","last_undegraded":"2024-09-15T23:37:14.011492+0000","last_fullsized":"2024-09-15T23:37:14.011492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.025069+0000","last_change":"2024-09-15T23:37:16.023795+0000","last_active":"2024-09-15T23:37:16.025069+0000","last_peered":"2024-09-15T23:37:16.025069+0000","last_clean":"2024-09-15T23:37:16.025069+0000","last_became_active":"2024-09-15T23:37:16.023493+0000","last_became_peered":"2024-09-15T23:37:16.023493+0000","last_unstale":"2024-09-15T23:37:16.025069+0000","last_undegraded":"2024-09-15T23:37:16.025069+0000","last_fullsized":"2024-09-15T23:37:16.025069+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:19.252190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.011446+0000","last_change":"2024-09-15T23:37:14.011446+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.011446+0000","last_undegraded":"2024-09-15T23:37:14.011446+0000","last_fullsized":"2024-09-15T23:37:14.011446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.022413+0000","last_change":"2024-09-15T23:37:15.020340+0000","last_active":"2024-09-15T23:37:15.022413+0000","last_peered":"2024-09-15T23:37:15.022413+0000","last_clean":"2024-09-15T23:37:15.022413+0000","last_became_active":"2024-09-15T23:37:15.020031+0000","last_became_peered":"2024-09-15T23:37:15.020031+0000","last_unstale":"2024-09-15T23:37:15.022413+0000","last_undegraded":"2024-09-15T23:37:15.022413+0000","last_fullsized":"2024-09-15T23:37:15.022413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:50.440261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051420+0000","last_change":"2024-09-15T23:37:16.049420+0000","last_active":"2024-09-15T23:37:16.051420+0000","last_peered":"2024-09-15T23:37:16.051420+0000","last_clean":"2024-09-15T23:37:16.051420+0000","last_became_active":"2024-09-15T23:37:16.049125+0000","last_became_peered":"2024-09-15T23:37:16.049125+0000","last_unstale":"2024-09-15T23:37:16.051420+0000","last_undegraded":"2024-09-15T23:37:16.051420+0000","last_fullsized":"2024-09-15T23:37:16.051420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:56.834193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.024060+0000","last_change":"2024-09-15T23:37:15.030423+0000","last_active":"2024-09-15T23:37:16.024060+0000","last_peered":"2024-09-15T23:37:16.024060+0000","last_clean":"2024-09-15T23:37:16.024060+0000","last_became_active":"2024-09-15T23:37:15.030077+0000","last_became_peered":"2024-09-15T23:37:15.030077+0000","last_unstale":"2024-09-15T23:37:16.024060+0000","last_undegraded":"2024-09-15T23:37:16.024060+0000","last_fullsized":"2024-09-15T23:37:16.024060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.262938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.021265+0000","last_change":"2024-09-15T23:37:15.018265+0000","last_active":"2024-09-15T23:37:15.021265+0000","last_peered":"2024-09-15T23:37:15.021265+0000","last_clean":"2024-09-15T23:37:15.021265+0000","last_became_active":"2024-09-15T23:37:15.017961+0000","last_became_peered":"2024-09-15T23:37:15.017961+0000","last_unstale":"2024-09-15T23:37:15.021265+0000","last_undegraded":"2024-09-15T23:37:15.021265+0000","last_fullsized":"2024-09-15T23:37:15.021265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:36:13.583601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014174+0000","last_change":"2024-09-15T23:37:15.014174+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014174+0000","last_undegraded":"2024-09-15T23:37:15.014174+0000","last_fullsized":"2024-09-15T23:37:15.014174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052977+0000","last_change":"2024-09-15T23:37:15.021730+0000","last_active":"2024-09-15T23:37:16.052977+0000","last_peered":"2024-09-15T23:37:16.052977+0000","last_clean":"2024-09-15T23:37:16.052977+0000","last_became_active":"2024-09-15T23:37:15.021217+0000","last_became_peered":"2024-09-15T23:37:15.021217+0000","last_unstale":"2024-09-15T23:37:16.052977+0000","last_undegraded":"2024-09-15T23:37:16.052977+0000","last_fullsized":"2024-09-15T23:37:16.052977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:43:54.440674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052881+0000","last_change":"2024-09-15T23:37:16.050557+0000","last_active":"2024-09-15T23:37:16.052881+0000","last_peered":"2024-09-15T23:37:16.052881+0000","last_clean":"2024-09-15T23:37:16.052881+0000","last_became_active":"2024-09-15T23:37:16.050267+0000","last_became_peered":"2024-09-15T23:37:16.050267+0000","last_unstale":"2024-09-15T23:37:16.052881+0000","last_undegraded":"2024-09-15T23:37:16.052881+0000","last_fullsized":"2024-09-15T23:37:16.052881+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:01.512554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.030092+0000","last_change":"2024-09-15T23:37:15.029187+0000","last_active":"2024-09-15T23:37:15.030092+0000","last_peered":"2024-09-15T23:37:15.030092+0000","last_clean":"2024-09-15T23:37:15.030092+0000","last_became_active":"2024-09-15T23:37:15.028731+0000","last_became_peered":"2024-09-15T23:37:15.028731+0000","last_unstale":"2024-09-15T23:37:15.030092+0000","last_undegraded":"2024-09-15T23:37:15.030092+0000","last_fullsized":"2024-09-15T23:37:15.030092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.262938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.029676+0000","last_change":"2024-09-15T23:37:15.028765+0000","last_active":"2024-09-15T23:37:15.029676+0000","last_peered":"2024-09-15T23:37:15.029676+0000","last_clean":"2024-09-15T23:37:15.029676+0000","last_became_active":"2024-09-15T23:37:15.028426+0000","last_became_peered":"2024-09-15T23:37:15.028426+0000","last_unstale":"2024-09-15T23:37:15.029676+0000","last_undegraded":"2024-09-15T23:37:15.029676+0000","last_fullsized":"2024-09-15T23:37:15.029676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:32.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.012677+0000","last_change":"2024-09-15T23:37:15.012677+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.012677+0000","last_undegraded":"2024-09-15T23:37:15.012677+0000","last_fullsized":"2024-09-15T23:37:15.012677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008364+0000","last_change":"2024-09-15T23:37:14.008364+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008364+0000","last_undegraded":"2024-09-15T23:37:14.008364+0000","last_fullsized":"2024-09-15T23:37:14.008364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013767+0000","last_change":"2024-09-15T23:37:15.013767+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013767+0000","last_undegraded":"2024-09-15T23:37:15.013767+0000","last_fullsized":"2024-09-15T23:37:15.013767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.026104+0000","last_change":"2024-09-15T23:37:15.023867+0000","last_active":"2024-09-15T23:37:16.026104+0000","last_peered":"2024-09-15T23:37:16.026104+0000","last_clean":"2024-09-15T23:37:16.026104+0000","last_became_active":"2024-09-15T23:37:15.023470+0000","last_became_peered":"2024-09-15T23:37:15.023470+0000","last_unstale":"2024-09-15T23:37:16.026104+0000","last_undegraded":"2024-09-15T23:37:16.026104+0000","last_fullsized":"2024-09-15T23:37:16.026104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:13.335910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044234+0000","last_change":"2024-09-15T23:37:16.041472+0000","last_active":"2024-09-15T23:37:16.044234+0000","last_peered":"2024-09-15T23:37:16.044234+0000","last_clean":"2024-09-15T23:37:16.044234+0000","last_became_active":"2024-09-15T23:37:16.040586+0000","last_became_peered":"2024-09-15T23:37:16.040586+0000","last_unstale":"2024-09-15T23:37:16.044234+0000","last_undegraded":"2024-09-15T23:37:16.044234+0000","last_fullsized":"2024-09-15T23:37:16.044234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:01.946016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020562+0000","last_change":"2024-09-15T23:37:15.022433+0000","last_active":"2024-09-15T23:37:16.020562+0000","last_peered":"2024-09-15T23:37:16.020562+0000","last_clean":"2024-09-15T23:37:16.020562+0000","last_became_active":"2024-09-15T23:37:15.022150+0000","last_became_peered":"2024-09-15T23:37:15.022150+0000","last_unstale":"2024-09-15T23:37:16.020562+0000","last_undegraded":"2024-09-15T23:37:16.020562+0000","last_fullsized":"2024-09-15T23:37:16.020562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:06.425003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033731+0000","last_change":"2024-09-15T23:37:16.031076+0000","last_active":"2024-09-15T23:37:16.033731+0000","last_peered":"2024-09-15T23:37:16.033731+0000","last_clean":"2024-09-15T23:37:16.033731+0000","last_became_active":"2024-09-15T23:37:16.030718+0000","last_became_peered":"2024-09-15T23:37:16.030718+0000","last_unstale":"2024-09-15T23:37:16.033731+0000","last_undegraded":"2024-09-15T23:37:16.033731+0000","last_fullsized":"2024-09-15T23:37:16.033731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:31.411462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008468+0000","last_change":"2024-09-15T23:37:14.008468+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008468+0000","last_undegraded":"2024-09-15T23:37:14.008468+0000","last_fullsized":"2024-09-15T23:37:14.008468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035661+0000","last_change":"2024-09-15T23:37:16.034326+0000","last_active":"2024-09-15T23:37:16.035661+0000","last_peered":"2024-09-15T23:37:16.035661+0000","last_clean":"2024-09-15T23:37:16.035661+0000","last_became_active":"2024-09-15T23:37:16.033729+0000","last_became_peered":"2024-09-15T23:37:16.033729+0000","last_unstale":"2024-09-15T23:37:16.035661+0000","last_undegraded":"2024-09-15T23:37:16.035661+0000","last_fullsized":"2024-09-15T23:37:16.035661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:50.331476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.027426+0000","last_change":"2024-09-15T23:37:15.024501+0000","last_active":"2024-09-15T23:37:16.027426+0000","last_peered":"2024-09-15T23:37:16.027426+0000","last_clean":"2024-09-15T23:37:16.027426+0000","last_became_active":"2024-09-15T23:37:15.024195+0000","last_became_peered":"2024-09-15T23:37:15.024195+0000","last_unstale":"2024-09-15T23:37:16.027426+0000","last_undegraded":"2024-09-15T23:37:16.027426+0000","last_fullsized":"2024-09-15T23:37:16.027426+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:31.148391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022665+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:37:16.022665+0000","last_peered":"2024-09-15T23:37:16.022665+0000","last_clean":"2024-09-15T23:37:16.022665+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:16.022665+0000","last_undegraded":"2024-09-15T23:37:16.022665+0000","last_fullsized":"2024-09-15T23:37:16.022665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":185007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":446464,"data_stored":187201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1028096,"data_stored":775397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":438272,"data_stored":185099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":417792,"data_stored":177840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":417792,"data_stored":177840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":761541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":171173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:37:19.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:37:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.498+0000 7feb9cfe4640 1 Processor -- start 2024-09-15T23:37:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.498+0000 7feb9cfe4640 1 -- start start 2024-09-15T23:37:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.499+0000 7feb9cfe4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.499+0000 7feb9cfe4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7feb980d32a0 0x7feb980b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.499+0000 7feb9cfe4640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb980b3d10 con 0x7feb980d51c0 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.499+0000 7feb9cfe4640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7feb980cfcd0 con 0x7feb980d32a0 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.499+0000 7feb9cfe4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7feb980cff50 con 0x7feb9815e6d0 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.499+0000 7feb95d9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7feb980d51c0 0x7feb980d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51940/0 (socket says 172.21.15.7:51940) 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.499+0000 7feb95d9b640 1 -- 172.21.15.7:0/2913922853 learned_addr learned my addr 172.21.15.7:0/2913922853 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb96d9d640 1 --2- 172.21.15.7:0/2913922853 >> v2:172.21.15.18:3300/0 conn(0x7feb980d32a0 0x7feb980b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb9659c640 1 --2- 172.21.15.7:0/2913922853 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb9559a640 1 -- 172.21.15.7:0/2913922853 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2328974205 0 0) 0x7feb980b3d10 con 0x7feb980d51c0 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb9559a640 1 -- 172.21.15.7:0/2913922853 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb84003520 con 0x7feb980d51c0 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb9559a640 1 -- 172.21.15.7:0/2913922853 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2974339271 0 0) 0x7feb84003520 con 0x7feb980d51c0 2024-09-15T23:37:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb96d9d640 1 -- 172.21.15.7:0/2913922853 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 msgr2=0x7feb980d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb96d9d640 1 --2- 172.21.15.7:0/2913922853 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb96d9d640 1 -- 172.21.15.7:0/2913922853 >> v1:172.21.15.7:6789/0 conn(0x7feb980d51c0 legacy=0x7feb980d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb96d9d640 1 -- 172.21.15.7:0/2913922853 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb980d0230 con 0x7feb980d32a0 2024-09-15T23:37:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.500+0000 7feb9659c640 1 --2- 172.21.15.7:0/2913922853 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.501+0000 7feb96d9d640 1 --2- 172.21.15.7:0/2913922853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb980d32a0 0x7feb980b41c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feb900097a0 tx=0x7feb9002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85d6e861a90b6583 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.501+0000 7feb9559a640 1 -- 172.21.15.7:0/2913922853 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90037070 con 0x7feb980d32a0 2024-09-15T23:37:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.501+0000 7feb9559a640 1 -- 172.21.15.7:0/2913922853 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9002aba0 con 0x7feb980d32a0 2024-09-15T23:37:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.501+0000 7feb9559a640 1 -- 172.21.15.7:0/2913922853 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb800016c0 con 0x7feb980d32a0 2024-09-15T23:37:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.501+0000 7feb9cfe4640 1 -- 172.21.15.7:0/2913922853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb980d32a0 msgr2=0x7feb980b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.501+0000 7feb9cfe4640 1 --2- 172.21.15.7:0/2913922853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb980d32a0 0x7feb980b41c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feb900097a0 tx=0x7feb9002a110 comp rx=0 tx=0).stop 2024-09-15T23:37:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.502+0000 7feb9cfe4640 1 -- 172.21.15.7:0/2913922853 shutdown_connections 2024-09-15T23:37:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.502+0000 7feb9cfe4640 1 --2- 172.21.15.7:0/2913922853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb980d32a0 0x7feb980b41c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.502+0000 7feb9cfe4640 1 --2- 172.21.15.7:0/2913922853 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.502+0000 7feb9cfe4640 1 -- 172.21.15.7:0/2913922853 >> 172.21.15.7:0/2913922853 conn(0x7feb9808bfd0 msgr2=0x7feb980d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.502+0000 7feb9cfe4640 1 -- 172.21.15.7:0/2913922853 shutdown_connections 2024-09-15T23:37:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.502+0000 7feb9cfe4640 1 -- 172.21.15.7:0/2913922853 wait complete. 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.503+0000 7feb9cfe4640 1 Processor -- start 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.503+0000 7feb9cfe4640 1 -- start start 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb9cfe4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb980d32a0 0x7feb980c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb9cfe4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb980d51c0 0x7feb981549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb9cfe4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb9cfe4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feb980d26f0 con 0x7feb980d51c0 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb9cfe4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feb980d0510 con 0x7feb980d32a0 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb9cfe4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7feb980d2870 con 0x7feb9815e6d0 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb95d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb980d51c0 0x7feb981549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb95d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb980d51c0 0x7feb981549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54510/0 (socket says 172.21.15.7:54510) 2024-09-15T23:37:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb95d9b640 1 -- 172.21.15.7:0/3359860275 learned_addr learned my addr 172.21.15.7:0/3359860275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb9659c640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb980d32a0 0x7feb980c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb96d9d640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb95d9b640 1 -- 172.21.15.7:0/3359860275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 msgr2=0x7feb980ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb95d9b640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb95d9b640 1 -- 172.21.15.7:0/3359860275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb980d32a0 msgr2=0x7feb980c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb95d9b640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb980d32a0 0x7feb980c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb95d9b640 1 -- 172.21.15.7:0/3359860275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb980ca7d0 con 0x7feb980d51c0 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb96d9d640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.504+0000 7feb95d9b640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb980d51c0 0x7feb981549f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feb8000c700 tx=0x7feb8000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.505+0000 7feb8effd640 1 -- 172.21.15.7:0/3359860275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80005ad0 con 0x7feb980d51c0 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.505+0000 7feb8effd640 1 -- 172.21.15.7:0/3359860275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb80005c70 con 0x7feb980d51c0 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.505+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb980caa50 con 0x7feb980d51c0 2024-09-15T23:37:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.505+0000 7feb8effd640 1 -- 172.21.15.7:0/3359860275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80005e60 con 0x7feb980d51c0 2024-09-15T23:37:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.505+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb980cb010 con 0x7feb980d51c0 2024-09-15T23:37:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.506+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb980d21b0 con 0x7feb980d51c0 2024-09-15T23:37:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.506+0000 7feb8effd640 1 -- 172.21.15.7:0/3359860275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feb80007290 con 0x7feb980d51c0 2024-09-15T23:37:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.507+0000 7feb8effd640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7feb7806c2b0 0x7feb7806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.507+0000 7feb8effd640 1 -- 172.21.15.7:0/3359860275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7feb8008b490 con 0x7feb980d51c0 2024-09-15T23:37:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.507+0000 7feb9659c640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7feb7806c2b0 0x7feb7806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.507+0000 7feb9659c640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7feb7806c2b0 0x7feb7806e770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feb88007440 tx=0x7feb8800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:19.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.511+0000 7feb8effd640 1 -- 172.21.15.7:0/3359860275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feb980ca7d0 con 0x7feb980d51c0 2024-09-15T23:37:19.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.683+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb980d2c10 con 0x7feb7806c2b0 2024-09-15T23:37:19.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.689+0000 7feb8effd640 1 -- 172.21.15.7:0/3359860275 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247699 (secure 0 0 0) 0x7feb980d2c10 con 0x7feb7806c2b0 2024-09-15T23:37:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:19.693 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:37:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7feb7806c2b0 msgr2=0x7feb7806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7feb7806c2b0 0x7feb7806e770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feb88007440 tx=0x7feb8800b040 comp rx=0 tx=0).stop 2024-09-15T23:37:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb980d51c0 msgr2=0x7feb981549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb980d51c0 0x7feb981549f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feb8000c700 tx=0x7feb8000cbc0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 shutdown_connections 2024-09-15T23:37:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7feb7806c2b0 0x7feb7806e770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feb9815e6d0 0x7feb980ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb980d51c0 0x7feb981549f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 --2- 172.21.15.7:0/3359860275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb980d32a0 0x7feb980c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 >> 172.21.15.7:0/3359860275 conn(0x7feb9808bfd0 msgr2=0x7feb980d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 shutdown_connections 2024-09-15T23:37:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.692+0000 7feb9cfe4640 1 -- 172.21.15.7:0/3359860275 wait complete. 2024-09-15T23:37:19.704 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-15T23:37:17.823024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254004,"kb_used_data":4476,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720716,"statfs":{"total":773094113280,"available":772834013184,"internally_reserved":0,"allocated":4583424,"data_stored":2621098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000643"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014716+0000","last_change":"2024-09-15T23:37:15.014716+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014716+0000","last_undegraded":"2024-09-15T23:37:15.014716+0000","last_fullsized":"2024-09-15T23:37:15.014716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013817+0000","last_change":"2024-09-15T23:37:15.013817+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013817+0000","last_undegraded":"2024-09-15T23:37:15.013817+0000","last_fullsized":"2024-09-15T23:37:15.013817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015854+0000","last_change":"2024-09-15T23:37:15.015854+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015854+0000","last_undegraded":"2024-09-15T23:37:15.015854+0000","last_fullsized":"2024-09-15T23:37:15.015854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048204+0000","last_change":"2024-09-15T23:37:16.039624+0000","last_active":"2024-09-15T23:37:16.048204+0000","last_peered":"2024-09-15T23:37:16.048204+0000","last_clean":"2024-09-15T23:37:16.048204+0000","last_became_active":"2024-09-15T23:37:16.038879+0000","last_became_peered":"2024-09-15T23:37:16.038879+0000","last_unstale":"2024-09-15T23:37:16.048204+0000","last_undegraded":"2024-09-15T23:37:16.048204+0000","last_fullsized":"2024-09-15T23:37:16.048204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:51.545065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013261+0000","last_change":"2024-09-15T23:37:15.013261+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013261+0000","last_undegraded":"2024-09-15T23:37:15.013261+0000","last_fullsized":"2024-09-15T23:37:15.013261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051987+0000","last_change":"2024-09-15T23:37:16.043510+0000","last_active":"2024-09-15T23:37:16.051987+0000","last_peered":"2024-09-15T23:37:16.051987+0000","last_clean":"2024-09-15T23:37:16.051987+0000","last_became_active":"2024-09-15T23:37:16.043208+0000","last_became_peered":"2024-09-15T23:37:16.043208+0000","last_unstale":"2024-09-15T23:37:16.051987+0000","last_undegraded":"2024-09-15T23:37:16.051987+0000","last_fullsized":"2024-09-15T23:37:16.051987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:10.099889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052859+0000","last_change":"2024-09-15T23:37:16.050348+0000","last_active":"2024-09-15T23:37:16.052859+0000","last_peered":"2024-09-15T23:37:16.052859+0000","last_clean":"2024-09-15T23:37:16.052859+0000","last_became_active":"2024-09-15T23:37:16.049871+0000","last_became_peered":"2024-09-15T23:37:16.049871+0000","last_unstale":"2024-09-15T23:37:16.052859+0000","last_undegraded":"2024-09-15T23:37:16.052859+0000","last_fullsized":"2024-09-15T23:37:16.052859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:23.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015421+0000","last_change":"2024-09-15T23:37:15.015421+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015421+0000","last_undegraded":"2024-09-15T23:37:15.015421+0000","last_fullsized":"2024-09-15T23:37:15.015421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047271+0000","last_change":"2024-09-15T23:37:16.037928+0000","last_active":"2024-09-15T23:37:16.047271+0000","last_peered":"2024-09-15T23:37:16.047271+0000","last_clean":"2024-09-15T23:37:16.047271+0000","last_became_active":"2024-09-15T23:37:16.037645+0000","last_became_peered":"2024-09-15T23:37:16.037645+0000","last_unstale":"2024-09-15T23:37:16.047271+0000","last_undegraded":"2024-09-15T23:37:16.047271+0000","last_fullsized":"2024-09-15T23:37:16.047271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:29.660110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048427+0000","last_change":"2024-09-15T23:37:16.046081+0000","last_active":"2024-09-15T23:37:16.048427+0000","last_peered":"2024-09-15T23:37:16.048427+0000","last_clean":"2024-09-15T23:37:16.048427+0000","last_became_active":"2024-09-15T23:37:16.045733+0000","last_became_peered":"2024-09-15T23:37:16.045733+0000","last_unstale":"2024-09-15T23:37:16.048427+0000","last_undegraded":"2024-09-15T23:37:16.048427+0000","last_fullsized":"2024-09-15T23:37:16.048427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036156+0000","last_change":"2024-09-15T23:37:16.024098+0000","last_active":"2024-09-15T23:37:16.036156+0000","last_peered":"2024-09-15T23:37:16.036156+0000","last_clean":"2024-09-15T23:37:16.036156+0000","last_became_active":"2024-09-15T23:37:16.023753+0000","last_became_peered":"2024-09-15T23:37:16.023753+0000","last_unstale":"2024-09-15T23:37:16.036156+0000","last_undegraded":"2024-09-15T23:37:16.036156+0000","last_fullsized":"2024-09-15T23:37:16.036156+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:34.586227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052091+0000","last_change":"2024-09-15T23:37:16.048043+0000","last_active":"2024-09-15T23:37:16.052091+0000","last_peered":"2024-09-15T23:37:16.052091+0000","last_clean":"2024-09-15T23:37:16.052091+0000","last_became_active":"2024-09-15T23:37:16.047587+0000","last_became_peered":"2024-09-15T23:37:16.047587+0000","last_unstale":"2024-09-15T23:37:16.052091+0000","last_undegraded":"2024-09-15T23:37:16.052091+0000","last_fullsized":"2024-09-15T23:37:16.052091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:57.383248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035550+0000","last_change":"2024-09-15T23:37:16.034026+0000","last_active":"2024-09-15T23:37:16.035550+0000","last_peered":"2024-09-15T23:37:16.035550+0000","last_clean":"2024-09-15T23:37:16.035550+0000","last_became_active":"2024-09-15T23:37:16.033385+0000","last_became_peered":"2024-09-15T23:37:16.033385+0000","last_unstale":"2024-09-15T23:37:16.035550+0000","last_undegraded":"2024-09-15T23:37:16.035550+0000","last_fullsized":"2024-09-15T23:37:16.035550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:33.923553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035623+0000","last_change":"2024-09-15T23:37:16.034152+0000","last_active":"2024-09-15T23:37:16.035623+0000","last_peered":"2024-09-15T23:37:16.035623+0000","last_clean":"2024-09-15T23:37:16.035623+0000","last_became_active":"2024-09-15T23:37:16.029607+0000","last_became_peered":"2024-09-15T23:37:16.029607+0000","last_unstale":"2024-09-15T23:37:16.035623+0000","last_undegraded":"2024-09-15T23:37:16.035623+0000","last_fullsized":"2024-09-15T23:37:16.035623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:55.516820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042577+0000","last_change":"2024-09-15T23:37:16.037756+0000","last_active":"2024-09-15T23:37:16.042577+0000","last_peered":"2024-09-15T23:37:16.042577+0000","last_clean":"2024-09-15T23:37:16.042577+0000","last_became_active":"2024-09-15T23:37:16.037309+0000","last_became_peered":"2024-09-15T23:37:16.037309+0000","last_unstale":"2024-09-15T23:37:16.042577+0000","last_undegraded":"2024-09-15T23:37:16.042577+0000","last_fullsized":"2024-09-15T23:37:16.042577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:45.986835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042641+0000","last_change":"2024-09-15T23:37:16.037039+0000","last_active":"2024-09-15T23:37:16.042641+0000","last_peered":"2024-09-15T23:37:16.042641+0000","last_clean":"2024-09-15T23:37:16.042641+0000","last_became_active":"2024-09-15T23:37:16.036774+0000","last_became_peered":"2024-09-15T23:37:16.036774+0000","last_unstale":"2024-09-15T23:37:16.042641+0000","last_undegraded":"2024-09-15T23:37:16.042641+0000","last_fullsized":"2024-09-15T23:37:16.042641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:30.804394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.040576+0000","last_change":"2024-09-15T23:37:16.034353+0000","last_active":"2024-09-15T23:37:16.040576+0000","last_peered":"2024-09-15T23:37:16.040576+0000","last_clean":"2024-09-15T23:37:16.040576+0000","last_became_active":"2024-09-15T23:37:16.033979+0000","last_became_peered":"2024-09-15T23:37:16.033979+0000","last_unstale":"2024-09-15T23:37:16.040576+0000","last_undegraded":"2024-09-15T23:37:16.040576+0000","last_fullsized":"2024-09-15T23:37:16.040576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:05.128368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036134+0000","last_change":"2024-09-15T23:37:16.027056+0000","last_active":"2024-09-15T23:37:16.036134+0000","last_peered":"2024-09-15T23:37:16.036134+0000","last_clean":"2024-09-15T23:37:16.036134+0000","last_became_active":"2024-09-15T23:37:16.026780+0000","last_became_peered":"2024-09-15T23:37:16.026780+0000","last_unstale":"2024-09-15T23:37:16.036134+0000","last_undegraded":"2024-09-15T23:37:16.036134+0000","last_fullsized":"2024-09-15T23:37:16.036134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:43.279890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044498+0000","last_change":"2024-09-15T23:37:16.042298+0000","last_active":"2024-09-15T23:37:16.044498+0000","last_peered":"2024-09-15T23:37:16.044498+0000","last_clean":"2024-09-15T23:37:16.044498+0000","last_became_active":"2024-09-15T23:37:16.041961+0000","last_became_peered":"2024-09-15T23:37:16.041961+0000","last_unstale":"2024-09-15T23:37:16.044498+0000","last_undegraded":"2024-09-15T23:37:16.044498+0000","last_fullsized":"2024-09-15T23:37:16.044498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:14.340846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033699+0000","last_change":"2024-09-15T23:37:16.031292+0000","last_active":"2024-09-15T23:37:16.033699+0000","last_peered":"2024-09-15T23:37:16.033699+0000","last_clean":"2024-09-15T23:37:16.033699+0000","last_became_active":"2024-09-15T23:37:16.030975+0000","last_became_peered":"2024-09-15T23:37:16.030975+0000","last_unstale":"2024-09-15T23:37:16.033699+0000","last_undegraded":"2024-09-15T23:37:16.033699+0000","last_fullsized":"2024-09-15T23:37:16.033699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:12.360351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048310+0000","last_change":"2024-09-15T23:37:16.039927+0000","last_active":"2024-09-15T23:37:16.048310+0000","last_peered":"2024-09-15T23:37:16.048310+0000","last_clean":"2024-09-15T23:37:16.048310+0000","last_became_active":"2024-09-15T23:37:16.039622+0000","last_became_peered":"2024-09-15T23:37:16.039622+0000","last_unstale":"2024-09-15T23:37:16.048310+0000","last_undegraded":"2024-09-15T23:37:16.048310+0000","last_fullsized":"2024-09-15T23:37:16.048310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:54.978065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035781+0000","last_change":"2024-09-15T23:37:16.032318+0000","last_active":"2024-09-15T23:37:16.035781+0000","last_peered":"2024-09-15T23:37:16.035781+0000","last_clean":"2024-09-15T23:37:16.035781+0000","last_became_active":"2024-09-15T23:37:16.028787+0000","last_became_peered":"2024-09-15T23:37:16.028787+0000","last_unstale":"2024-09-15T23:37:16.035781+0000","last_undegraded":"2024-09-15T23:37:16.035781+0000","last_fullsized":"2024-09-15T23:37:16.035781+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:31.065835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035685+0000","last_change":"2024-09-15T23:37:16.028365+0000","last_active":"2024-09-15T23:37:16.035685+0000","last_peered":"2024-09-15T23:37:16.035685+0000","last_clean":"2024-09-15T23:37:16.035685+0000","last_became_active":"2024-09-15T23:37:16.028051+0000","last_became_peered":"2024-09-15T23:37:16.028051+0000","last_unstale":"2024-09-15T23:37:16.035685+0000","last_undegraded":"2024-09-15T23:37:16.035685+0000","last_fullsized":"2024-09-15T23:37:16.035685+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:47.439397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048225+0000","last_change":"2024-09-15T23:37:16.039870+0000","last_active":"2024-09-15T23:37:16.048225+0000","last_peered":"2024-09-15T23:37:16.048225+0000","last_clean":"2024-09-15T23:37:16.048225+0000","last_became_active":"2024-09-15T23:37:16.039295+0000","last_became_peered":"2024-09-15T23:37:16.039295+0000","last_unstale":"2024-09-15T23:37:16.048225+0000","last_undegraded":"2024-09-15T23:37:16.048225+0000","last_fullsized":"2024-09-15T23:37:16.048225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:16.959191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046178+0000","last_change":"2024-09-15T23:37:16.044114+0000","last_active":"2024-09-15T23:37:16.046178+0000","last_peered":"2024-09-15T23:37:16.046178+0000","last_clean":"2024-09-15T23:37:16.046178+0000","last_became_active":"2024-09-15T23:37:16.042005+0000","last_became_peered":"2024-09-15T23:37:16.042005+0000","last_unstale":"2024-09-15T23:37:16.046178+0000","last_undegraded":"2024-09-15T23:37:16.046178+0000","last_fullsized":"2024-09-15T23:37:16.046178+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:40.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056140+0000","last_change":"2024-09-15T23:37:15.019375+0000","last_active":"2024-09-15T23:37:16.056140+0000","last_peered":"2024-09-15T23:37:16.056140+0000","last_clean":"2024-09-15T23:37:16.056140+0000","last_became_active":"2024-09-15T23:37:15.018977+0000","last_became_peered":"2024-09-15T23:37:15.018977+0000","last_unstale":"2024-09-15T23:37:16.056140+0000","last_undegraded":"2024-09-15T23:37:16.056140+0000","last_fullsized":"2024-09-15T23:37:16.056140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:02.654673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013153+0000","last_change":"2024-09-15T23:37:15.013153+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013153+0000","last_undegraded":"2024-09-15T23:37:15.013153+0000","last_fullsized":"2024-09-15T23:37:15.013153+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.027944+0000","last_change":"2024-09-15T23:37:15.024812+0000","last_active":"2024-09-15T23:37:15.027944+0000","last_peered":"2024-09-15T23:37:15.027944+0000","last_clean":"2024-09-15T23:37:15.027944+0000","last_became_active":"2024-09-15T23:37:15.024533+0000","last_became_peered":"2024-09-15T23:37:15.024533+0000","last_unstale":"2024-09-15T23:37:15.027944+0000","last_undegraded":"2024-09-15T23:37:15.027944+0000","last_fullsized":"2024-09-15T23:37:15.027944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:56.378313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046817+0000","last_change":"2024-09-15T23:37:16.043209+0000","last_active":"2024-09-15T23:37:16.046817+0000","last_peered":"2024-09-15T23:37:16.046817+0000","last_clean":"2024-09-15T23:37:16.046817+0000","last_became_active":"2024-09-15T23:37:16.042016+0000","last_became_peered":"2024-09-15T23:37:16.042016+0000","last_unstale":"2024-09-15T23:37:16.046817+0000","last_undegraded":"2024-09-15T23:37:16.046817+0000","last_fullsized":"2024-09-15T23:37:16.046817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:17.167815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008364+0000","last_change":"2024-09-15T23:37:14.008364+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008364+0000","last_undegraded":"2024-09-15T23:37:14.008364+0000","last_fullsized":"2024-09-15T23:37:14.008364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014181+0000","last_change":"2024-09-15T23:37:15.014181+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014181+0000","last_undegraded":"2024-09-15T23:37:15.014181+0000","last_fullsized":"2024-09-15T23:37:15.014181+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056405+0000","last_change":"2024-09-15T23:37:15.023109+0000","last_active":"2024-09-15T23:37:16.056405+0000","last_peered":"2024-09-15T23:37:16.056405+0000","last_clean":"2024-09-15T23:37:16.056405+0000","last_became_active":"2024-09-15T23:37:15.022494+0000","last_became_peered":"2024-09-15T23:37:15.022494+0000","last_unstale":"2024-09-15T23:37:16.056405+0000","last_undegraded":"2024-09-15T23:37:16.056405+0000","last_fullsized":"2024-09-15T23:37:16.056405+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:29.332641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.055917+0000","last_change":"2024-09-15T23:37:15.025209+0000","last_active":"2024-09-15T23:37:16.055917+0000","last_peered":"2024-09-15T23:37:16.055917+0000","last_clean":"2024-09-15T23:37:16.055917+0000","last_became_active":"2024-09-15T23:37:15.022678+0000","last_became_peered":"2024-09-15T23:37:15.022678+0000","last_unstale":"2024-09-15T23:37:16.055917+0000","last_undegraded":"2024-09-15T23:37:16.055917+0000","last_fullsized":"2024-09-15T23:37:16.055917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:01.000785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033653+0000","last_change":"2024-09-15T23:37:16.031990+0000","last_active":"2024-09-15T23:37:16.033653+0000","last_peered":"2024-09-15T23:37:16.033653+0000","last_clean":"2024-09-15T23:37:16.033653+0000","last_became_active":"2024-09-15T23:37:16.031688+0000","last_became_peered":"2024-09-15T23:37:16.031688+0000","last_unstale":"2024-09-15T23:37:16.033653+0000","last_undegraded":"2024-09-15T23:37:16.033653+0000","last_fullsized":"2024-09-15T23:37:16.033653+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:36.654534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052263+0000","last_change":"2024-09-15T23:37:15.029001+0000","last_active":"2024-09-15T23:37:16.052263+0000","last_peered":"2024-09-15T23:37:16.052263+0000","last_clean":"2024-09-15T23:37:16.052263+0000","last_became_active":"2024-09-15T23:37:15.028669+0000","last_became_peered":"2024-09-15T23:37:15.028669+0000","last_unstale":"2024-09-15T23:37:16.052263+0000","last_undegraded":"2024-09-15T23:37:16.052263+0000","last_fullsized":"2024-09-15T23:37:16.052263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:32.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035709+0000","last_change":"2024-09-15T23:37:16.034125+0000","last_active":"2024-09-15T23:37:16.035709+0000","last_peered":"2024-09-15T23:37:16.035709+0000","last_clean":"2024-09-15T23:37:16.035709+0000","last_became_active":"2024-09-15T23:37:16.033801+0000","last_became_peered":"2024-09-15T23:37:16.033801+0000","last_unstale":"2024-09-15T23:37:16.035709+0000","last_undegraded":"2024-09-15T23:37:16.035709+0000","last_fullsized":"2024-09-15T23:37:16.035709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:04.589184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008354+0000","last_change":"2024-09-15T23:37:14.008354+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008354+0000","last_undegraded":"2024-09-15T23:37:14.008354+0000","last_fullsized":"2024-09-15T23:37:14.008354+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013255+0000","last_change":"2024-09-15T23:37:15.013255+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013255+0000","last_undegraded":"2024-09-15T23:37:15.013255+0000","last_fullsized":"2024-09-15T23:37:15.013255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022594+0000","last_change":"2024-09-15T23:37:15.026937+0000","last_active":"2024-09-15T23:37:16.022594+0000","last_peered":"2024-09-15T23:37:16.022594+0000","last_clean":"2024-09-15T23:37:16.022594+0000","last_became_active":"2024-09-15T23:37:15.026293+0000","last_became_peered":"2024-09-15T23:37:15.026293+0000","last_unstale":"2024-09-15T23:37:16.022594+0000","last_undegraded":"2024-09-15T23:37:16.022594+0000","last_fullsized":"2024-09-15T23:37:16.022594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:29.524303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008569+0000","last_change":"2024-09-15T23:37:14.008569+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008569+0000","last_undegraded":"2024-09-15T23:37:14.008569+0000","last_fullsized":"2024-09-15T23:37:14.008569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.045715+0000","last_change":"2024-09-15T23:37:16.044875+0000","last_active":"2024-09-15T23:37:16.045715+0000","last_peered":"2024-09-15T23:37:16.045715+0000","last_clean":"2024-09-15T23:37:16.045715+0000","last_became_active":"2024-09-15T23:37:16.044600+0000","last_became_peered":"2024-09-15T23:37:16.044600+0000","last_unstale":"2024-09-15T23:37:16.045715+0000","last_undegraded":"2024-09-15T23:37:16.045715+0000","last_fullsized":"2024-09-15T23:37:16.045715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:32.153327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020234+0000","last_change":"2024-09-15T23:37:15.025268+0000","last_active":"2024-09-15T23:37:16.020234+0000","last_peered":"2024-09-15T23:37:16.020234+0000","last_clean":"2024-09-15T23:37:16.020234+0000","last_became_active":"2024-09-15T23:37:15.024965+0000","last_became_peered":"2024-09-15T23:37:15.024965+0000","last_unstale":"2024-09-15T23:37:16.020234+0000","last_undegraded":"2024-09-15T23:37:16.020234+0000","last_fullsized":"2024-09-15T23:37:16.020234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:41.312641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.053070+0000","last_change":"2024-09-15T23:37:15.025534+0000","last_active":"2024-09-15T23:37:16.053070+0000","last_peered":"2024-09-15T23:37:16.053070+0000","last_clean":"2024-09-15T23:37:16.053070+0000","last_became_active":"2024-09-15T23:37:15.023762+0000","last_became_peered":"2024-09-15T23:37:15.023762+0000","last_unstale":"2024-09-15T23:37:16.053070+0000","last_undegraded":"2024-09-15T23:37:16.053070+0000","last_fullsized":"2024-09-15T23:37:16.053070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:11.328653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051853+0000","last_change":"2024-09-15T23:37:16.043056+0000","last_active":"2024-09-15T23:37:16.051853+0000","last_peered":"2024-09-15T23:37:16.051853+0000","last_clean":"2024-09-15T23:37:16.051853+0000","last_became_active":"2024-09-15T23:37:16.042639+0000","last_became_peered":"2024-09-15T23:37:16.042639+0000","last_unstale":"2024-09-15T23:37:16.051853+0000","last_undegraded":"2024-09-15T23:37:16.051853+0000","last_fullsized":"2024-09-15T23:37:16.051853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:51.445196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008657+0000","last_change":"2024-09-15T23:37:14.008657+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008657+0000","last_undegraded":"2024-09-15T23:37:14.008657+0000","last_fullsized":"2024-09-15T23:37:14.008657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.009242+0000","last_change":"2024-09-15T23:37:14.009242+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.009242+0000","last_undegraded":"2024-09-15T23:37:14.009242+0000","last_fullsized":"2024-09-15T23:37:14.009242+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015544+0000","last_change":"2024-09-15T23:37:15.015544+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015544+0000","last_undegraded":"2024-09-15T23:37:15.015544+0000","last_fullsized":"2024-09-15T23:37:15.015544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.023511+0000","last_change":"2024-09-15T23:37:15.021306+0000","last_active":"2024-09-15T23:37:15.023511+0000","last_peered":"2024-09-15T23:37:15.023511+0000","last_clean":"2024-09-15T23:37:15.023511+0000","last_became_active":"2024-09-15T23:37:15.020973+0000","last_became_peered":"2024-09-15T23:37:15.020973+0000","last_unstale":"2024-09-15T23:37:15.023511+0000","last_undegraded":"2024-09-15T23:37:15.023511+0000","last_fullsized":"2024-09-15T23:37:15.023511+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:09.094953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.034232+0000","last_change":"2024-09-15T23:37:16.028516+0000","last_active":"2024-09-15T23:37:16.034232+0000","last_peered":"2024-09-15T23:37:16.034232+0000","last_clean":"2024-09-15T23:37:16.034232+0000","last_became_active":"2024-09-15T23:37:16.028203+0000","last_became_peered":"2024-09-15T23:37:16.028203+0000","last_unstale":"2024-09-15T23:37:16.034232+0000","last_undegraded":"2024-09-15T23:37:16.034232+0000","last_fullsized":"2024-09-15T23:37:16.034232+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020739+0000","last_change":"2024-09-15T23:37:15.020887+0000","last_active":"2024-09-15T23:37:16.020739+0000","last_peered":"2024-09-15T23:37:16.020739+0000","last_clean":"2024-09-15T23:37:16.020739+0000","last_became_active":"2024-09-15T23:37:15.020611+0000","last_became_peered":"2024-09-15T23:37:15.020611+0000","last_unstale":"2024-09-15T23:37:16.020739+0000","last_undegraded":"2024-09-15T23:37:16.020739+0000","last_fullsized":"2024-09-15T23:37:16.020739+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:55.272773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013244+0000","last_change":"2024-09-15T23:37:15.013244+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013244+0000","last_undegraded":"2024-09-15T23:37:15.013244+0000","last_fullsized":"2024-09-15T23:37:15.013244+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.058124+0000","last_change":"2024-09-15T23:37:15.018587+0000","last_active":"2024-09-15T23:37:16.058124+0000","last_peered":"2024-09-15T23:37:16.058124+0000","last_clean":"2024-09-15T23:37:16.058124+0000","last_became_active":"2024-09-15T23:37:15.018296+0000","last_became_peered":"2024-09-15T23:37:15.018296+0000","last_unstale":"2024-09-15T23:37:16.058124+0000","last_undegraded":"2024-09-15T23:37:16.058124+0000","last_fullsized":"2024-09-15T23:37:16.058124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:01.784529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013616+0000","last_change":"2024-09-15T23:37:15.013616+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013616+0000","last_undegraded":"2024-09-15T23:37:15.013616+0000","last_fullsized":"2024-09-15T23:37:15.013616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052687+0000","last_change":"2024-09-15T23:37:15.025903+0000","last_active":"2024-09-15T23:37:16.052687+0000","last_peered":"2024-09-15T23:37:16.052687+0000","last_clean":"2024-09-15T23:37:16.052687+0000","last_became_active":"2024-09-15T23:37:15.025212+0000","last_became_peered":"2024-09-15T23:37:15.025212+0000","last_unstale":"2024-09-15T23:37:16.052687+0000","last_undegraded":"2024-09-15T23:37:16.052687+0000","last_fullsized":"2024-09-15T23:37:16.052687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:41.537186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047929+0000","last_change":"2024-09-15T23:37:16.042373+0000","last_active":"2024-09-15T23:37:16.047929+0000","last_peered":"2024-09-15T23:37:16.047929+0000","last_clean":"2024-09-15T23:37:16.047929+0000","last_became_active":"2024-09-15T23:37:16.040111+0000","last_became_peered":"2024-09-15T23:37:16.040111+0000","last_unstale":"2024-09-15T23:37:16.047929+0000","last_undegraded":"2024-09-15T23:37:16.047929+0000","last_fullsized":"2024-09-15T23:37:16.047929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:36:14.588537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052462+0000","last_change":"2024-09-15T23:37:15.024606+0000","last_active":"2024-09-15T23:37:16.052462+0000","last_peered":"2024-09-15T23:37:16.052462+0000","last_clean":"2024-09-15T23:37:16.052462+0000","last_became_active":"2024-09-15T23:37:15.024349+0000","last_became_peered":"2024-09-15T23:37:15.024349+0000","last_unstale":"2024-09-15T23:37:16.052462+0000","last_undegraded":"2024-09-15T23:37:16.052462+0000","last_fullsized":"2024-09-15T23:37:16.052462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:23.692350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047894+0000","last_change":"2024-09-15T23:37:16.041862+0000","last_active":"2024-09-15T23:37:16.047894+0000","last_peered":"2024-09-15T23:37:16.047894+0000","last_clean":"2024-09-15T23:37:16.047894+0000","last_became_active":"2024-09-15T23:37:16.041522+0000","last_became_peered":"2024-09-15T23:37:16.041522+0000","last_unstale":"2024-09-15T23:37:16.047894+0000","last_undegraded":"2024-09-15T23:37:16.047894+0000","last_fullsized":"2024-09-15T23:37:16.047894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:08.290613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.025129+0000","last_change":"2024-09-15T23:37:15.023432+0000","last_active":"2024-09-15T23:37:15.025129+0000","last_peered":"2024-09-15T23:37:15.025129+0000","last_clean":"2024-09-15T23:37:15.025129+0000","last_became_active":"2024-09-15T23:37:15.023142+0000","last_became_peered":"2024-09-15T23:37:15.023142+0000","last_unstale":"2024-09-15T23:37:15.025129+0000","last_undegraded":"2024-09-15T23:37:15.025129+0000","last_fullsized":"2024-09-15T23:37:15.025129+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:41.537186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.011492+0000","last_change":"2024-09-15T23:37:14.011492+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.011492+0000","last_undegraded":"2024-09-15T23:37:14.011492+0000","last_fullsized":"2024-09-15T23:37:14.011492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.025069+0000","last_change":"2024-09-15T23:37:16.023795+0000","last_active":"2024-09-15T23:37:16.025069+0000","last_peered":"2024-09-15T23:37:16.025069+0000","last_clean":"2024-09-15T23:37:16.025069+0000","last_became_active":"2024-09-15T23:37:16.023493+0000","last_became_peered":"2024-09-15T23:37:16.023493+0000","last_unstale":"2024-09-15T23:37:16.025069+0000","last_undegraded":"2024-09-15T23:37:16.025069+0000","last_fullsized":"2024-09-15T23:37:16.025069+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:19.252190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.011446+0000","last_change":"2024-09-15T23:37:14.011446+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.011446+0000","last_undegraded":"2024-09-15T23:37:14.011446+0000","last_fullsized":"2024-09-15T23:37:14.011446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.022413+0000","last_change":"2024-09-15T23:37:15.020340+0000","last_active":"2024-09-15T23:37:15.022413+0000","last_peered":"2024-09-15T23:37:15.022413+0000","last_clean":"2024-09-15T23:37:15.022413+0000","last_became_active":"2024-09-15T23:37:15.020031+0000","last_became_peered":"2024-09-15T23:37:15.020031+0000","last_unstale":"2024-09-15T23:37:15.022413+0000","last_undegraded":"2024-09-15T23:37:15.022413+0000","last_fullsized":"2024-09-15T23:37:15.022413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:50.440261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051420+0000","last_change":"2024-09-15T23:37:16.049420+0000","last_active":"2024-09-15T23:37:16.051420+0000","last_peered":"2024-09-15T23:37:16.051420+0000","last_clean":"2024-09-15T23:37:16.051420+0000","last_became_active":"2024-09-15T23:37:16.049125+0000","last_became_peered":"2024-09-15T23:37:16.049125+0000","last_unstale":"2024-09-15T23:37:16.051420+0000","last_undegraded":"2024-09-15T23:37:16.051420+0000","last_fullsized":"2024-09-15T23:37:16.051420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:56.834193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.024060+0000","last_change":"2024-09-15T23:37:15.030423+0000","last_active":"2024-09-15T23:37:16.024060+0000","last_peered":"2024-09-15T23:37:16.024060+0000","last_clean":"2024-09-15T23:37:16.024060+0000","last_became_active":"2024-09-15T23:37:15.030077+0000","last_became_peered":"2024-09-15T23:37:15.030077+0000","last_unstale":"2024-09-15T23:37:16.024060+0000","last_undegraded":"2024-09-15T23:37:16.024060+0000","last_fullsized":"2024-09-15T23:37:16.024060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.262938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.021265+0000","last_change":"2024-09-15T23:37:15.018265+0000","last_active":"2024-09-15T23:37:15.021265+0000","last_peered":"2024-09-15T23:37:15.021265+0000","last_clean":"2024-09-15T23:37:15.021265+0000","last_became_active":"2024-09-15T23:37:15.017961+0000","last_became_peered":"2024-09-15T23:37:15.017961+0000","last_unstale":"2024-09-15T23:37:15.021265+0000","last_undegraded":"2024-09-15T23:37:15.021265+0000","last_fullsized":"2024-09-15T23:37:15.021265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:36:13.583601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014174+0000","last_change":"2024-09-15T23:37:15.014174+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014174+0000","last_undegraded":"2024-09-15T23:37:15.014174+0000","last_fullsized":"2024-09-15T23:37:15.014174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052977+0000","last_change":"2024-09-15T23:37:15.021730+0000","last_active":"2024-09-15T23:37:16.052977+0000","last_peered":"2024-09-15T23:37:16.052977+0000","last_clean":"2024-09-15T23:37:16.052977+0000","last_became_active":"2024-09-15T23:37:15.021217+0000","last_became_peered":"2024-09-15T23:37:15.021217+0000","last_unstale":"2024-09-15T23:37:16.052977+0000","last_undegraded":"2024-09-15T23:37:16.052977+0000","last_fullsized":"2024-09-15T23:37:16.052977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:43:54.440674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052881+0000","last_change":"2024-09-15T23:37:16.050557+0000","last_active":"2024-09-15T23:37:16.052881+0000","last_peered":"2024-09-15T23:37:16.052881+0000","last_clean":"2024-09-15T23:37:16.052881+0000","last_became_active":"2024-09-15T23:37:16.050267+0000","last_became_peered":"2024-09-15T23:37:16.050267+0000","last_unstale":"2024-09-15T23:37:16.052881+0000","last_undegraded":"2024-09-15T23:37:16.052881+0000","last_fullsized":"2024-09-15T23:37:16.052881+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:01.512554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.030092+0000","last_change":"2024-09-15T23:37:15.029187+0000","last_active":"2024-09-15T23:37:15.030092+0000","last_peered":"2024-09-15T23:37:15.030092+0000","last_clean":"2024-09-15T23:37:15.030092+0000","last_became_active":"2024-09-15T23:37:15.028731+0000","last_became_peered":"2024-09-15T23:37:15.028731+0000","last_unstale":"2024-09-15T23:37:15.030092+0000","last_undegraded":"2024-09-15T23:37:15.030092+0000","last_fullsized":"2024-09-15T23:37:15.030092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.262938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.029676+0000","last_change":"2024-09-15T23:37:15.028765+0000","last_active":"2024-09-15T23:37:15.029676+0000","last_peered":"2024-09-15T23:37:15.029676+0000","last_clean":"2024-09-15T23:37:15.029676+0000","last_became_active":"2024-09-15T23:37:15.028426+0000","last_became_peered":"2024-09-15T23:37:15.028426+0000","last_unstale":"2024-09-15T23:37:15.029676+0000","last_undegraded":"2024-09-15T23:37:15.029676+0000","last_fullsized":"2024-09-15T23:37:15.029676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:32.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.012677+0000","last_change":"2024-09-15T23:37:15.012677+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.012677+0000","last_undegraded":"2024-09-15T23:37:15.012677+0000","last_fullsized":"2024-09-15T23:37:15.012677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008364+0000","last_change":"2024-09-15T23:37:14.008364+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008364+0000","last_undegraded":"2024-09-15T23:37:14.008364+0000","last_fullsized":"2024-09-15T23:37:14.008364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013767+0000","last_change":"2024-09-15T23:37:15.013767+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013767+0000","last_undegraded":"2024-09-15T23:37:15.013767+0000","last_fullsized":"2024-09-15T23:37:15.013767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.026104+0000","last_change":"2024-09-15T23:37:15.023867+0000","last_active":"2024-09-15T23:37:16.026104+0000","last_peered":"2024-09-15T23:37:16.026104+0000","last_clean":"2024-09-15T23:37:16.026104+0000","last_became_active":"2024-09-15T23:37:15.023470+0000","last_became_peered":"2024-09-15T23:37:15.023470+0000","last_unstale":"2024-09-15T23:37:16.026104+0000","last_undegraded":"2024-09-15T23:37:16.026104+0000","last_fullsized":"2024-09-15T23:37:16.026104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:13.335910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044234+0000","last_change":"2024-09-15T23:37:16.041472+0000","last_active":"2024-09-15T23:37:16.044234+0000","last_peered":"2024-09-15T23:37:16.044234+0000","last_clean":"2024-09-15T23:37:16.044234+0000","last_became_active":"2024-09-15T23:37:16.040586+0000","last_became_peered":"2024-09-15T23:37:16.040586+0000","last_unstale":"2024-09-15T23:37:16.044234+0000","last_undegraded":"2024-09-15T23:37:16.044234+0000","last_fullsized":"2024-09-15T23:37:16.044234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:01.946016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020562+0000","last_change":"2024-09-15T23:37:15.022433+0000","last_active":"2024-09-15T23:37:16.020562+0000","last_peered":"2024-09-15T23:37:16.020562+0000","last_clean":"2024-09-15T23:37:16.020562+0000","last_became_active":"2024-09-15T23:37:15.022150+0000","last_became_peered":"2024-09-15T23:37:15.022150+0000","last_unstale":"2024-09-15T23:37:16.020562+0000","last_undegraded":"2024-09-15T23:37:16.020562+0000","last_fullsized":"2024-09-15T23:37:16.020562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:06.425003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033731+0000","last_change":"2024-09-15T23:37:16.031076+0000","last_active":"2024-09-15T23:37:16.033731+0000","last_peered":"2024-09-15T23:37:16.033731+0000","last_clean":"2024-09-15T23:37:16.033731+0000","last_became_active":"2024-09-15T23:37:16.030718+0000","last_became_peered":"2024-09-15T23:37:16.030718+0000","last_unstale":"2024-09-15T23:37:16.033731+0000","last_undegraded":"2024-09-15T23:37:16.033731+0000","last_fullsized":"2024-09-15T23:37:16.033731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:31.411462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008468+0000","last_change":"2024-09-15T23:37:14.008468+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008468+0000","last_undegraded":"2024-09-15T23:37:14.008468+0000","last_fullsized":"2024-09-15T23:37:14.008468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035661+0000","last_change":"2024-09-15T23:37:16.034326+0000","last_active":"2024-09-15T23:37:16.035661+0000","last_peered":"2024-09-15T23:37:16.035661+0000","last_clean":"2024-09-15T23:37:16.035661+0000","last_became_active":"2024-09-15T23:37:16.033729+0000","last_became_peered":"2024-09-15T23:37:16.033729+0000","last_unstale":"2024-09-15T23:37:16.035661+0000","last_undegraded":"2024-09-15T23:37:16.035661+0000","last_fullsized":"2024-09-15T23:37:16.035661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:50.331476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.027426+0000","last_change":"2024-09-15T23:37:15.024501+0000","last_active":"2024-09-15T23:37:16.027426+0000","last_peered":"2024-09-15T23:37:16.027426+0000","last_clean":"2024-09-15T23:37:16.027426+0000","last_became_active":"2024-09-15T23:37:15.024195+0000","last_became_peered":"2024-09-15T23:37:15.024195+0000","last_unstale":"2024-09-15T23:37:16.027426+0000","last_undegraded":"2024-09-15T23:37:16.027426+0000","last_fullsized":"2024-09-15T23:37:16.027426+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:31.148391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022665+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:37:16.022665+0000","last_peered":"2024-09-15T23:37:16.022665+0000","last_clean":"2024-09-15T23:37:16.022665+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:16.022665+0000","last_undegraded":"2024-09-15T23:37:16.022665+0000","last_fullsized":"2024-09-15T23:37:16.022665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":185007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":446464,"data_stored":187201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1028096,"data_stored":775397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":438272,"data_stored":185099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":417792,"data_stored":177840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":417792,"data_stored":177840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":761541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":171173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:37:19.707 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3f is not active+clean 2024-09-15T23:37:19.707 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3e is not active+clean 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.014716+0000', 'last_change': '2024-09-15T23:37:15.014716+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.014716+0000', 'last_undegraded': '2024-09-15T23:37:15.014716+0000', 'last_fullsized': '2024-09-15T23:37:15.014716+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3c is not active+clean 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3b is not active+clean 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.013817+0000', 'last_change': '2024-09-15T23:37:15.013817+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.013817+0000', 'last_undegraded': '2024-09-15T23:37:15.013817+0000', 'last_fullsized': '2024-09-15T23:37:15.013817+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3a is not active+clean 2024-09-15T23:37:19.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.015854+0000', 'last_change': '2024-09-15T23:37:15.015854+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.015854+0000', 'last_undegraded': '2024-09-15T23:37:15.015854+0000', 'last_fullsized': '2024-09-15T23:37:15.015854+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:PG 6.38 is not active+clean 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.013261+0000', 'last_change': '2024-09-15T23:37:15.013261+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.013261+0000', 'last_undegraded': '2024-09-15T23:37:15.013261+0000', 'last_fullsized': '2024-09-15T23:37:15.013261+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.015421+0000', 'last_change': '2024-09-15T23:37:15.015421+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.015421+0000', 'last_undegraded': '2024-09-15T23:37:15.015421+0000', 'last_fullsized': '2024-09-15T23:37:15.015421+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2d is not active+clean 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:PG 6.28 is not active+clean 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.709 INFO:tasks.ceph.ceph_manager.ceph:PG 6.26 is not active+clean 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:PG 6.20 is not active+clean 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1d is not active+clean 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.013153+0000', 'last_change': '2024-09-15T23:37:15.013153+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.013153+0000', 'last_undegraded': '2024-09-15T23:37:15.013153+0000', 'last_fullsized': '2024-09-15T23:37:15.013153+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1d is not active+clean 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:14.008364+0000', 'last_change': '2024-09-15T23:37:14.008364+0000', 'last_active': '2024-09-15T23:37:13.998584+0000', 'last_peered': '2024-09-15T23:37:13.998584+0000', 'last_clean': '2024-09-15T23:37:13.998584+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:14.008364+0000', 'last_undegraded': '2024-09-15T23:37:14.008364+0000', 'last_fullsized': '2024-09-15T23:37:14.008364+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1f is not active+clean 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.014181+0000', 'last_change': '2024-09-15T23:37:15.014181+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.014181+0000', 'last_undegraded': '2024-09-15T23:37:15.014181+0000', 'last_fullsized': '2024-09-15T23:37:15.014181+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-09-15T23:37:19.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:PG 5.19 is not active+clean 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:14.008354+0000', 'last_change': '2024-09-15T23:37:14.008354+0000', 'last_active': '2024-09-15T23:37:13.998584+0000', 'last_peered': '2024-09-15T23:37:13.998584+0000', 'last_clean': '2024-09-15T23:37:13.998584+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:14.008354+0000', 'last_undegraded': '2024-09-15T23:37:14.008354+0000', 'last_fullsized': '2024-09-15T23:37:14.008354+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1b is not active+clean 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.013255+0000', 'last_change': '2024-09-15T23:37:15.013255+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.013255+0000', 'last_undegraded': '2024-09-15T23:37:15.013255+0000', 'last_fullsized': '2024-09-15T23:37:15.013255+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:PG 6.14 is not active+clean 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:PG 5.17 is not active+clean 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:14.008569+0000', 'last_change': '2024-09-15T23:37:14.008569+0000', 'last_active': '2024-09-15T23:37:13.998584+0000', 'last_peered': '2024-09-15T23:37:13.998584+0000', 'last_clean': '2024-09-15T23:37:13.998584+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:14.008569+0000', 'last_undegraded': '2024-09-15T23:37:14.008569+0000', 'last_fullsized': '2024-09-15T23:37:14.008569+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:PG 5.14 is not active+clean 2024-09-15T23:37:19.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:14.008657+0000', 'last_change': '2024-09-15T23:37:14.008657+0000', 'last_active': '2024-09-15T23:37:13.998584+0000', 'last_peered': '2024-09-15T23:37:13.998584+0000', 'last_clean': '2024-09-15T23:37:13.998584+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:14.008657+0000', 'last_undegraded': '2024-09-15T23:37:14.008657+0000', 'last_fullsized': '2024-09-15T23:37:14.008657+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T23:37:19.712 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-09-15T23:37:19.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.712 INFO:tasks.ceph.ceph_manager.ceph:PG 5.13 is not active+clean 2024-09-15T23:37:19.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:14.009242+0000', 'last_change': '2024-09-15T23:37:14.009242+0000', 'last_active': '2024-09-15T23:37:13.998584+0000', 'last_peered': '2024-09-15T23:37:13.998584+0000', 'last_clean': '2024-09-15T23:37:13.998584+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:14.009242+0000', 'last_undegraded': '2024-09-15T23:37:14.009242+0000', 'last_fullsized': '2024-09-15T23:37:14.009242+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T23:37:19.712 INFO:tasks.ceph.ceph_manager.ceph:PG 6.11 is not active+clean 2024-09-15T23:37:19.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.015544+0000', 'last_change': '2024-09-15T23:37:15.015544+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.015544+0000', 'last_undegraded': '2024-09-15T23:37:15.015544+0000', 'last_fullsized': '2024-09-15T23:37:15.015544+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T23:37:19.712 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.013244+0000', 'last_change': '2024-09-15T23:37:15.013244+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.013244+0000', 'last_undegraded': '2024-09-15T23:37:15.013244+0000', 'last_fullsized': '2024-09-15T23:37:15.013244+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:PG 6.c is not active+clean 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.013616+0000', 'last_change': '2024-09-15T23:37:15.013616+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.013616+0000', 'last_undegraded': '2024-09-15T23:37:15.013616+0000', 'last_fullsized': '2024-09-15T23:37:15.013616+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:PG 6.f is not active+clean 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:PG 5.c is not active+clean 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:14.011492+0000', 'last_change': '2024-09-15T23:37:14.011492+0000', 'last_active': '2024-09-15T23:37:13.998584+0000', 'last_peered': '2024-09-15T23:37:13.998584+0000', 'last_clean': '2024-09-15T23:37:13.998584+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:14.011492+0000', 'last_undegraded': '2024-09-15T23:37:14.011492+0000', 'last_fullsized': '2024-09-15T23:37:14.011492+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:14.011446+0000', 'last_change': '2024-09-15T23:37:14.011446+0000', 'last_active': '2024-09-15T23:37:13.998584+0000', 'last_peered': '2024-09-15T23:37:13.998584+0000', 'last_clean': '2024-09-15T23:37:13.998584+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:14.011446+0000', 'last_undegraded': '2024-09-15T23:37:14.011446+0000', 'last_fullsized': '2024-09-15T23:37:14.011446+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.713 INFO:tasks.ceph.ceph_manager.ceph:PG 6.b is not active+clean 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:PG 6.4 is not active+clean 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.014174+0000', 'last_change': '2024-09-15T23:37:15.014174+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.014174+0000', 'last_undegraded': '2024-09-15T23:37:15.014174+0000', 'last_fullsized': '2024-09-15T23:37:15.014174+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-15T23:37:15.003519+0000', 'last_change': '2024-09-15T23:37:15.003519+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.003519+0000', 'last_undegraded': '2024-09-15T23:37:15.003519+0000', 'last_fullsized': '2024-09-15T23:37:15.003519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.012677+0000', 'last_change': '2024-09-15T23:37:15.012677+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.012677+0000', 'last_undegraded': '2024-09-15T23:37:15.012677+0000', 'last_fullsized': '2024-09-15T23:37:15.012677+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3 is not active+clean 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:14.008364+0000', 'last_change': '2024-09-15T23:37:14.008364+0000', 'last_active': '2024-09-15T23:37:13.998584+0000', 'last_peered': '2024-09-15T23:37:13.998584+0000', 'last_clean': '2024-09-15T23:37:13.998584+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:14.008364+0000', 'last_undegraded': '2024-09-15T23:37:14.008364+0000', 'last_fullsized': '2024-09-15T23:37:14.008364+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1 is not active+clean 2024-09-15T23:37:19.714 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:15.013767+0000', 'last_change': '2024-09-15T23:37:15.013767+0000', 'last_active': '2024-09-15T23:37:15.003519+0000', 'last_peered': '2024-09-15T23:37:15.003519+0000', 'last_clean': '2024-09-15T23:37:15.003519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:15.013767+0000', 'last_undegraded': '2024-09-15T23:37:15.013767+0000', 'last_fullsized': '2024-09-15T23:37:15.013767+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:15.003519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T23:37:19.715 INFO:tasks.ceph.ceph_manager.ceph:PG 5.0 is not active+clean 2024-09-15T23:37:19.715 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-15T23:37:14.008468+0000', 'last_change': '2024-09-15T23:37:14.008468+0000', 'last_active': '2024-09-15T23:37:13.998584+0000', 'last_peered': '2024-09-15T23:37:13.998584+0000', 'last_clean': '2024-09-15T23:37:13.998584+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T23:37:14.008468+0000', 'last_undegraded': '2024-09-15T23:37:14.008468+0000', 'last_fullsized': '2024-09-15T23:37:14.008468+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'last_clean_scrub_stamp': '2024-09-15T23:37:13.998584+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T23:37:19.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T23:37:19.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.845+0000 7fe1126c9640 1 Processor -- start 2024-09-15T23:37:19.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.845+0000 7fe1126c9640 1 -- start start 2024-09-15T23:37:19.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.846+0000 7fe1126c9640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c06bcf0 0x7fe10c0d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.846+0000 7fe1126c9640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe10c0d0ac0 0x7fe10c0b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.846+0000 7fe1126c9640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe10c15caf0 con 0x7fe10c15c710 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.846+0000 7fe1126c9640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe10c0b5760 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.846+0000 7fe1126c9640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe10c0b4b50 con 0x7fe10c06bcf0 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.846+0000 7fe110c66640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe10c0d0ac0 0x7fe10c0b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.846+0000 7fe110c66640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe10c0d0ac0 0x7fe10c0b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54518/0 (socket says 172.21.15.7:54518) 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.846+0000 7fe110c66640 1 -- 172.21.15.7:0/243128515 learned_addr learned my addr 172.21.15.7:0/243128515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.847+0000 7fe10b7fe640 1 --2- 172.21.15.7:0/243128515 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c06bcf0 0x7fe10c0d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.847+0000 7fe110c66640 1 -- 172.21.15.7:0/243128515 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c06bcf0 msgr2=0x7fe10c0d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.847+0000 7fe110c66640 1 --2- 172.21.15.7:0/243128515 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c06bcf0 0x7fe10c0d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.847+0000 7fe110c66640 1 -- 172.21.15.7:0/243128515 >> v1:172.21.15.18:6789/0 conn(0x7fe10c15c710 legacy=0x7fe10c0b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.847+0000 7fe110c66640 1 -- 172.21.15.7:0/243128515 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c0b4e30 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.847+0000 7fe110c66640 1 --2- 172.21.15.7:0/243128515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 0x7fe10c0b5bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe104009e50 tx=0x7fe10402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1286386409e0fd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe10affd640 1 -- 172.21.15.7:0/243128515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104036070 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe10affd640 1 -- 172.21.15.7:0/243128515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe104031600 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe10affd640 1 -- 172.21.15.7:0/243128515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040319d0 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe1126c9640 1 -- 172.21.15.7:0/243128515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 msgr2=0x7fe10c0b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe1126c9640 1 --2- 172.21.15.7:0/243128515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 0x7fe10c0b5bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe104009e50 tx=0x7fe10402a710 comp rx=0 tx=0).stop 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe1126c9640 1 -- 172.21.15.7:0/243128515 shutdown_connections 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe1126c9640 1 --2- 172.21.15.7:0/243128515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 0x7fe10c0b5bf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe1126c9640 1 --2- 172.21.15.7:0/243128515 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c06bcf0 0x7fe10c0d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe1126c9640 1 -- 172.21.15.7:0/243128515 >> 172.21.15.7:0/243128515 conn(0x7fe10c09aa40 msgr2=0x7fe10c0b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe1126c9640 1 -- 172.21.15.7:0/243128515 shutdown_connections 2024-09-15T23:37:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.848+0000 7fe1126c9640 1 -- 172.21.15.7:0/243128515 wait complete. 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.849+0000 7fe1126c9640 1 Processor -- start 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.850+0000 7fe1126c9640 1 -- start start 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.850+0000 7fe1126c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe10c06bcf0 0x7fe10c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.850+0000 7fe1126c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 0x7fe10c0c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.850+0000 7fe1126c9640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c15c710 0x7fe10c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.850+0000 7fe1126c9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c0d0f70 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.850+0000 7fe1126c9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c0b53f0 con 0x7fe10c06bcf0 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.850+0000 7fe1126c9640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe10c0b6130 con 0x7fe10c15c710 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.850+0000 7fe10b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 0x7fe10c0c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe10b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 0x7fe10c0c8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54520/0 (socket says 172.21.15.7:54520) 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe10b7fe640 1 -- 172.21.15.7:0/811197002 learned_addr learned my addr 172.21.15.7:0/811197002 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe110c66640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c15c710 0x7fe10c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe10bfff640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe10c06bcf0 0x7fe10c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe10b7fe640 1 -- 172.21.15.7:0/811197002 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c15c710 msgr2=0x7fe10c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe10b7fe640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c15c710 0x7fe10c0c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe10b7fe640 1 -- 172.21.15.7:0/811197002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe10c06bcf0 msgr2=0x7fe10c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe10b7fe640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe10c06bcf0 0x7fe10c0c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe10b7fe640 1 -- 172.21.15.7:0/811197002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c0b1f20 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe10b7fe640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 0x7fe10c0c8970 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe10000d7d0 tx=0x7fe10000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe108ff9640 1 -- 172.21.15.7:0/811197002 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10000b6c0 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe108ff9640 1 -- 172.21.15.7:0/811197002 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe100019070 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe108ff9640 1 -- 172.21.15.7:0/811197002 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10000bac0 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c0c9f10 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.851+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c0c98c0 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.852+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c0d0580 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.853+0000 7fe108ff9640 1 -- 172.21.15.7:0/811197002 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe10000bc60 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.853+0000 7fe108ff9640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe0f806c2b0 0x7fe0f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.853+0000 7fe108ff9640 1 -- 172.21.15.7:0/811197002 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe10008b960 con 0x7fe10c0d0ac0 2024-09-15T23:37:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.854+0000 7fe10bfff640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe0f806c2b0 0x7fe0f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.854+0000 7fe10bfff640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe0f806c2b0 0x7fe0f806e770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0016e0 tx=0x7fe0fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:19.858+0000 7fe108ff9640 1 -- 172.21.15.7:0/811197002 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe100059870 con 0x7fe10c0d0ac0 2024-09-15T23:37:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.101+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe10c1e4f00 con 0x7fe10c0d0ac0 2024-09-15T23:37:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.102+0000 7fe108ff9640 1 -- 172.21.15.7:0/811197002 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1763 (secure 0 0 0) 0x7fe10005db70 con 0x7fe10c0d0ac0 2024-09-15T23:37:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:20.103 INFO:teuthology.orchestra.run.smithi007.stdout:{"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":80,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1726443370,"num_in_osds":8,"osd_in_since":1726443361,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":58},{"state_name":"creating+peering","count":23},{"state_name":"unknown","count":16}],"num_pgs":97,"num_pools":3,"num_objects":14,"data_bytes":591062,"bytes_used":260100096,"bytes_avail":772834013184,"bytes_total":773094113280,"unknown_pgs_ratio":0.16494844853878021,"inactive_pgs_ratio":0.23711340129375458,"write_bytes_sec":527,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":15,"btime":"2024-09-15T23:37:18:028501+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4830}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T23:37:13.822441+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T23:37:20.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe0f806c2b0 msgr2=0x7fe0f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:20.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe0f806c2b0 0x7fe0f806e770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0016e0 tx=0x7fe0fc009040 comp rx=0 tx=0).stop 2024-09-15T23:37:20.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 msgr2=0x7fe10c0c8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:20.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 0x7fe10c0c8970 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe10000d7d0 tx=0x7fe10000dc90 comp rx=0 tx=0).stop 2024-09-15T23:37:20.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 shutdown_connections 2024-09-15T23:37:20.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe0f806c2b0 0x7fe0f806e770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe10c15c710 0x7fe10c0c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10c0d0ac0 0x7fe10c0c8970 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 --2- 172.21.15.7:0/811197002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe10c06bcf0 0x7fe10c0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 >> 172.21.15.7:0/811197002 conn(0x7fe10c09aa40 msgr2=0x7fe10c0d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 shutdown_connections 2024-09-15T23:37:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.104+0000 7fe1126c9640 1 -- 172.21.15.7:0/811197002 wait complete. 2024-09-15T23:37:20.115 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T23:37:20.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:37:20.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.216+0000 7f5a590f1640 1 Processor -- start 2024-09-15T23:37:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.217+0000 7f5a590f1640 1 -- start start 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.217+0000 7f5a590f1640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a5415e840 0x7f5a540d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.217+0000 7f5a590f1640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a540d0f90 0x7f5a540d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.217+0000 7f5a590f1640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a5406dd80 con 0x7f5a540d52e0 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.217+0000 7f5a590f1640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5a540d1940 con 0x7f5a540d0f90 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.217+0000 7f5a590f1640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5a540cfe90 con 0x7f5a5415e840 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.217+0000 7f5a5359e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a540d0f90 0x7f5a540d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.218+0000 7f5a5359e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a540d0f90 0x7f5a540d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54530/0 (socket says 172.21.15.7:54530) 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.218+0000 7f5a5359e640 1 -- 172.21.15.7:0/2832035594 learned_addr learned my addr 172.21.15.7:0/2832035594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.218+0000 7f5a5359e640 1 -- 172.21.15.7:0/2832035594 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a5415e840 msgr2=0x7f5a540d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.218+0000 7f5a5359e640 1 --2- 172.21.15.7:0/2832035594 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a5415e840 0x7f5a540d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.218+0000 7f5a5359e640 1 -- 172.21.15.7:0/2832035594 >> v1:172.21.15.18:6789/0 conn(0x7f5a540d52e0 legacy=0x7f5a5406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.218+0000 7f5a5359e640 1 -- 172.21.15.7:0/2832035594 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a540cf560 con 0x7f5a540d0f90 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a5359e640 1 --2- 172.21.15.7:0/2832035594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 0x7f5a540d1400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009e50 tx=0x7f5a4c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509f7467e6aa375c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a51d9b640 1 -- 172.21.15.7:0/2832035594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c036070 con 0x7f5a540d0f90 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a51d9b640 1 -- 172.21.15.7:0/2832035594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4c031600 con 0x7f5a540d0f90 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a51d9b640 1 -- 172.21.15.7:0/2832035594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0319d0 con 0x7f5a540d0f90 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a590f1640 1 -- 172.21.15.7:0/2832035594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 msgr2=0x7f5a540d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a590f1640 1 --2- 172.21.15.7:0/2832035594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 0x7f5a540d1400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009e50 tx=0x7f5a4c02a6c0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a590f1640 1 -- 172.21.15.7:0/2832035594 shutdown_connections 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a590f1640 1 --2- 172.21.15.7:0/2832035594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 0x7f5a540d1400 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a590f1640 1 --2- 172.21.15.7:0/2832035594 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a5415e840 0x7f5a540d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a590f1640 1 -- 172.21.15.7:0/2832035594 >> 172.21.15.7:0/2832035594 conn(0x7f5a5408bfd0 msgr2=0x7f5a540d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a590f1640 1 -- 172.21.15.7:0/2832035594 shutdown_connections 2024-09-15T23:37:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.219+0000 7f5a590f1640 1 -- 172.21.15.7:0/2832035594 wait complete. 2024-09-15T23:37:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.221+0000 7f5a590f1640 1 Processor -- start 2024-09-15T23:37:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.221+0000 7f5a590f1640 1 -- start start 2024-09-15T23:37:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.221+0000 7f5a590f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 0x7f5a541f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.221+0000 7f5a590f1640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a540d52e0 0x7f5a5414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.221+0000 7f5a590f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5415e840 0x7f5a541576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a590f1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5406e030 con 0x7f5a540d0f90 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a590f1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5a540cfb20 con 0x7f5a5415e840 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a590f1640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5415ec20 con 0x7f5a540d52e0 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a52d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 0x7f5a541f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a52d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 0x7f5a541f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54542/0 (socket says 172.21.15.7:54542) 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a52d9d640 1 -- 172.21.15.7:0/4162983760 learned_addr learned my addr 172.21.15.7:0/4162983760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a5359e640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5415e840 0x7f5a541576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a5259c640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a540d52e0 0x7f5a5414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a52d9d640 1 -- 172.21.15.7:0/4162983760 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a540d52e0 msgr2=0x7f5a5414a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a52d9d640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a540d52e0 0x7f5a5414a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a52d9d640 1 -- 172.21.15.7:0/4162983760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5415e840 msgr2=0x7f5a541576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:20.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a52d9d640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5415e840 0x7f5a541576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a52d9d640 1 -- 172.21.15.7:0/4162983760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a541f5600 con 0x7f5a540d0f90 2024-09-15T23:37:20.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a52d9d640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 0x7f5a541f50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b410 tx=0x7f5a4400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:20.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a3b7fe640 1 -- 172.21.15.7:0/4162983760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44015020 con 0x7f5a540d0f90 2024-09-15T23:37:20.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a3b7fe640 1 -- 172.21.15.7:0/4162983760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a44011850 con 0x7f5a540d0f90 2024-09-15T23:37:20.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5414af00 con 0x7f5a540d0f90 2024-09-15T23:37:20.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a3b7fe640 1 -- 172.21.15.7:0/4162983760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44011c40 con 0x7f5a540d0f90 2024-09-15T23:37:20.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.222+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a540b5780 con 0x7f5a540d0f90 2024-09-15T23:37:20.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.223+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a540d1400 con 0x7f5a540d0f90 2024-09-15T23:37:20.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.224+0000 7f5a3b7fe640 1 -- 172.21.15.7:0/4162983760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5a44018020 con 0x7f5a540d0f90 2024-09-15T23:37:20.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.224+0000 7f5a3b7fe640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a4006c2b0 0x7f5a4006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:20.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.224+0000 7f5a3b7fe640 1 -- 172.21.15.7:0/4162983760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5a4408b3c0 con 0x7f5a540d0f90 2024-09-15T23:37:20.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.225+0000 7f5a5259c640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a4006c2b0 0x7f5a4006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:20.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.225+0000 7f5a5259c640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a4006c2b0 0x7f5a4006e770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a54158420 tx=0x7f5a48007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:20.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.229+0000 7f5a3b7fe640 1 -- 172.21.15.7:0/4162983760 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5a4401f020 con 0x7f5a540d0f90 2024-09-15T23:37:20.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.401+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a540d25f0 con 0x7f5a4006c2b0 2024-09-15T23:37:20.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.407+0000 7f5a3b7fe640 1 -- 172.21.15.7:0/4162983760 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247699 (secure 0 0 0) 0x7f5a540d25f0 con 0x7f5a4006c2b0 2024-09-15T23:37:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:20.411 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:37:20.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a4006c2b0 msgr2=0x7f5a4006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:20.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a4006c2b0 0x7f5a4006e770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a54158420 tx=0x7f5a48007ea0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 msgr2=0x7f5a541f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 0x7f5a541f50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b410 tx=0x7f5a4400b8d0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 shutdown_connections 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5415e840 0x7f5a541576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f5a4006c2b0 0x7f5a4006e770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a540d52e0 0x7f5a5414a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 --2- 172.21.15.7:0/4162983760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540d0f90 0x7f5a541f50c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 >> 172.21.15.7:0/4162983760 conn(0x7f5a5408bfd0 msgr2=0x7f5a540b9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 shutdown_connections 2024-09-15T23:37:20.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:20.410+0000 7f5a590f1640 1 -- 172.21.15.7:0/4162983760 wait complete. 2024-09-15T23:37:20.422 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-15T23:37:19.823339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254004,"kb_used_data":4476,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720716,"statfs":{"total":773094113280,"available":772834013184,"internally_reserved":0,"allocated":4583424,"data_stored":2621098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.819092"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014716+0000","last_change":"2024-09-15T23:37:15.014716+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014716+0000","last_undegraded":"2024-09-15T23:37:15.014716+0000","last_fullsized":"2024-09-15T23:37:15.014716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013817+0000","last_change":"2024-09-15T23:37:15.013817+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013817+0000","last_undegraded":"2024-09-15T23:37:15.013817+0000","last_fullsized":"2024-09-15T23:37:15.013817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015854+0000","last_change":"2024-09-15T23:37:15.015854+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015854+0000","last_undegraded":"2024-09-15T23:37:15.015854+0000","last_fullsized":"2024-09-15T23:37:15.015854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048204+0000","last_change":"2024-09-15T23:37:16.039624+0000","last_active":"2024-09-15T23:37:16.048204+0000","last_peered":"2024-09-15T23:37:16.048204+0000","last_clean":"2024-09-15T23:37:16.048204+0000","last_became_active":"2024-09-15T23:37:16.038879+0000","last_became_peered":"2024-09-15T23:37:16.038879+0000","last_unstale":"2024-09-15T23:37:16.048204+0000","last_undegraded":"2024-09-15T23:37:16.048204+0000","last_fullsized":"2024-09-15T23:37:16.048204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:51.545065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013261+0000","last_change":"2024-09-15T23:37:15.013261+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013261+0000","last_undegraded":"2024-09-15T23:37:15.013261+0000","last_fullsized":"2024-09-15T23:37:15.013261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051987+0000","last_change":"2024-09-15T23:37:16.043510+0000","last_active":"2024-09-15T23:37:16.051987+0000","last_peered":"2024-09-15T23:37:16.051987+0000","last_clean":"2024-09-15T23:37:16.051987+0000","last_became_active":"2024-09-15T23:37:16.043208+0000","last_became_peered":"2024-09-15T23:37:16.043208+0000","last_unstale":"2024-09-15T23:37:16.051987+0000","last_undegraded":"2024-09-15T23:37:16.051987+0000","last_fullsized":"2024-09-15T23:37:16.051987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:10.099889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052859+0000","last_change":"2024-09-15T23:37:16.050348+0000","last_active":"2024-09-15T23:37:16.052859+0000","last_peered":"2024-09-15T23:37:16.052859+0000","last_clean":"2024-09-15T23:37:16.052859+0000","last_became_active":"2024-09-15T23:37:16.049871+0000","last_became_peered":"2024-09-15T23:37:16.049871+0000","last_unstale":"2024-09-15T23:37:16.052859+0000","last_undegraded":"2024-09-15T23:37:16.052859+0000","last_fullsized":"2024-09-15T23:37:16.052859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:23.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015421+0000","last_change":"2024-09-15T23:37:15.015421+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015421+0000","last_undegraded":"2024-09-15T23:37:15.015421+0000","last_fullsized":"2024-09-15T23:37:15.015421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047271+0000","last_change":"2024-09-15T23:37:16.037928+0000","last_active":"2024-09-15T23:37:16.047271+0000","last_peered":"2024-09-15T23:37:16.047271+0000","last_clean":"2024-09-15T23:37:16.047271+0000","last_became_active":"2024-09-15T23:37:16.037645+0000","last_became_peered":"2024-09-15T23:37:16.037645+0000","last_unstale":"2024-09-15T23:37:16.047271+0000","last_undegraded":"2024-09-15T23:37:16.047271+0000","last_fullsized":"2024-09-15T23:37:16.047271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:29.660110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048427+0000","last_change":"2024-09-15T23:37:16.046081+0000","last_active":"2024-09-15T23:37:16.048427+0000","last_peered":"2024-09-15T23:37:16.048427+0000","last_clean":"2024-09-15T23:37:16.048427+0000","last_became_active":"2024-09-15T23:37:16.045733+0000","last_became_peered":"2024-09-15T23:37:16.045733+0000","last_unstale":"2024-09-15T23:37:16.048427+0000","last_undegraded":"2024-09-15T23:37:16.048427+0000","last_fullsized":"2024-09-15T23:37:16.048427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036156+0000","last_change":"2024-09-15T23:37:16.024098+0000","last_active":"2024-09-15T23:37:16.036156+0000","last_peered":"2024-09-15T23:37:16.036156+0000","last_clean":"2024-09-15T23:37:16.036156+0000","last_became_active":"2024-09-15T23:37:16.023753+0000","last_became_peered":"2024-09-15T23:37:16.023753+0000","last_unstale":"2024-09-15T23:37:16.036156+0000","last_undegraded":"2024-09-15T23:37:16.036156+0000","last_fullsized":"2024-09-15T23:37:16.036156+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:34.586227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052091+0000","last_change":"2024-09-15T23:37:16.048043+0000","last_active":"2024-09-15T23:37:16.052091+0000","last_peered":"2024-09-15T23:37:16.052091+0000","last_clean":"2024-09-15T23:37:16.052091+0000","last_became_active":"2024-09-15T23:37:16.047587+0000","last_became_peered":"2024-09-15T23:37:16.047587+0000","last_unstale":"2024-09-15T23:37:16.052091+0000","last_undegraded":"2024-09-15T23:37:16.052091+0000","last_fullsized":"2024-09-15T23:37:16.052091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:57.383248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035550+0000","last_change":"2024-09-15T23:37:16.034026+0000","last_active":"2024-09-15T23:37:16.035550+0000","last_peered":"2024-09-15T23:37:16.035550+0000","last_clean":"2024-09-15T23:37:16.035550+0000","last_became_active":"2024-09-15T23:37:16.033385+0000","last_became_peered":"2024-09-15T23:37:16.033385+0000","last_unstale":"2024-09-15T23:37:16.035550+0000","last_undegraded":"2024-09-15T23:37:16.035550+0000","last_fullsized":"2024-09-15T23:37:16.035550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:33.923553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035623+0000","last_change":"2024-09-15T23:37:16.034152+0000","last_active":"2024-09-15T23:37:16.035623+0000","last_peered":"2024-09-15T23:37:16.035623+0000","last_clean":"2024-09-15T23:37:16.035623+0000","last_became_active":"2024-09-15T23:37:16.029607+0000","last_became_peered":"2024-09-15T23:37:16.029607+0000","last_unstale":"2024-09-15T23:37:16.035623+0000","last_undegraded":"2024-09-15T23:37:16.035623+0000","last_fullsized":"2024-09-15T23:37:16.035623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:55.516820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042577+0000","last_change":"2024-09-15T23:37:16.037756+0000","last_active":"2024-09-15T23:37:16.042577+0000","last_peered":"2024-09-15T23:37:16.042577+0000","last_clean":"2024-09-15T23:37:16.042577+0000","last_became_active":"2024-09-15T23:37:16.037309+0000","last_became_peered":"2024-09-15T23:37:16.037309+0000","last_unstale":"2024-09-15T23:37:16.042577+0000","last_undegraded":"2024-09-15T23:37:16.042577+0000","last_fullsized":"2024-09-15T23:37:16.042577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:45.986835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042641+0000","last_change":"2024-09-15T23:37:16.037039+0000","last_active":"2024-09-15T23:37:16.042641+0000","last_peered":"2024-09-15T23:37:16.042641+0000","last_clean":"2024-09-15T23:37:16.042641+0000","last_became_active":"2024-09-15T23:37:16.036774+0000","last_became_peered":"2024-09-15T23:37:16.036774+0000","last_unstale":"2024-09-15T23:37:16.042641+0000","last_undegraded":"2024-09-15T23:37:16.042641+0000","last_fullsized":"2024-09-15T23:37:16.042641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:30.804394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.040576+0000","last_change":"2024-09-15T23:37:16.034353+0000","last_active":"2024-09-15T23:37:16.040576+0000","last_peered":"2024-09-15T23:37:16.040576+0000","last_clean":"2024-09-15T23:37:16.040576+0000","last_became_active":"2024-09-15T23:37:16.033979+0000","last_became_peered":"2024-09-15T23:37:16.033979+0000","last_unstale":"2024-09-15T23:37:16.040576+0000","last_undegraded":"2024-09-15T23:37:16.040576+0000","last_fullsized":"2024-09-15T23:37:16.040576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:05.128368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036134+0000","last_change":"2024-09-15T23:37:16.027056+0000","last_active":"2024-09-15T23:37:16.036134+0000","last_peered":"2024-09-15T23:37:16.036134+0000","last_clean":"2024-09-15T23:37:16.036134+0000","last_became_active":"2024-09-15T23:37:16.026780+0000","last_became_peered":"2024-09-15T23:37:16.026780+0000","last_unstale":"2024-09-15T23:37:16.036134+0000","last_undegraded":"2024-09-15T23:37:16.036134+0000","last_fullsized":"2024-09-15T23:37:16.036134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:43.279890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044498+0000","last_change":"2024-09-15T23:37:16.042298+0000","last_active":"2024-09-15T23:37:16.044498+0000","last_peered":"2024-09-15T23:37:16.044498+0000","last_clean":"2024-09-15T23:37:16.044498+0000","last_became_active":"2024-09-15T23:37:16.041961+0000","last_became_peered":"2024-09-15T23:37:16.041961+0000","last_unstale":"2024-09-15T23:37:16.044498+0000","last_undegraded":"2024-09-15T23:37:16.044498+0000","last_fullsized":"2024-09-15T23:37:16.044498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:14.340846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033699+0000","last_change":"2024-09-15T23:37:16.031292+0000","last_active":"2024-09-15T23:37:16.033699+0000","last_peered":"2024-09-15T23:37:16.033699+0000","last_clean":"2024-09-15T23:37:16.033699+0000","last_became_active":"2024-09-15T23:37:16.030975+0000","last_became_peered":"2024-09-15T23:37:16.030975+0000","last_unstale":"2024-09-15T23:37:16.033699+0000","last_undegraded":"2024-09-15T23:37:16.033699+0000","last_fullsized":"2024-09-15T23:37:16.033699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:12.360351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048310+0000","last_change":"2024-09-15T23:37:16.039927+0000","last_active":"2024-09-15T23:37:16.048310+0000","last_peered":"2024-09-15T23:37:16.048310+0000","last_clean":"2024-09-15T23:37:16.048310+0000","last_became_active":"2024-09-15T23:37:16.039622+0000","last_became_peered":"2024-09-15T23:37:16.039622+0000","last_unstale":"2024-09-15T23:37:16.048310+0000","last_undegraded":"2024-09-15T23:37:16.048310+0000","last_fullsized":"2024-09-15T23:37:16.048310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:54.978065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035781+0000","last_change":"2024-09-15T23:37:16.032318+0000","last_active":"2024-09-15T23:37:16.035781+0000","last_peered":"2024-09-15T23:37:16.035781+0000","last_clean":"2024-09-15T23:37:16.035781+0000","last_became_active":"2024-09-15T23:37:16.028787+0000","last_became_peered":"2024-09-15T23:37:16.028787+0000","last_unstale":"2024-09-15T23:37:16.035781+0000","last_undegraded":"2024-09-15T23:37:16.035781+0000","last_fullsized":"2024-09-15T23:37:16.035781+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:31.065835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035685+0000","last_change":"2024-09-15T23:37:16.028365+0000","last_active":"2024-09-15T23:37:16.035685+0000","last_peered":"2024-09-15T23:37:16.035685+0000","last_clean":"2024-09-15T23:37:16.035685+0000","last_became_active":"2024-09-15T23:37:16.028051+0000","last_became_peered":"2024-09-15T23:37:16.028051+0000","last_unstale":"2024-09-15T23:37:16.035685+0000","last_undegraded":"2024-09-15T23:37:16.035685+0000","last_fullsized":"2024-09-15T23:37:16.035685+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:47.439397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048225+0000","last_change":"2024-09-15T23:37:16.039870+0000","last_active":"2024-09-15T23:37:16.048225+0000","last_peered":"2024-09-15T23:37:16.048225+0000","last_clean":"2024-09-15T23:37:16.048225+0000","last_became_active":"2024-09-15T23:37:16.039295+0000","last_became_peered":"2024-09-15T23:37:16.039295+0000","last_unstale":"2024-09-15T23:37:16.048225+0000","last_undegraded":"2024-09-15T23:37:16.048225+0000","last_fullsized":"2024-09-15T23:37:16.048225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:16.959191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046178+0000","last_change":"2024-09-15T23:37:16.044114+0000","last_active":"2024-09-15T23:37:16.046178+0000","last_peered":"2024-09-15T23:37:16.046178+0000","last_clean":"2024-09-15T23:37:16.046178+0000","last_became_active":"2024-09-15T23:37:16.042005+0000","last_became_peered":"2024-09-15T23:37:16.042005+0000","last_unstale":"2024-09-15T23:37:16.046178+0000","last_undegraded":"2024-09-15T23:37:16.046178+0000","last_fullsized":"2024-09-15T23:37:16.046178+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:40.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056140+0000","last_change":"2024-09-15T23:37:15.019375+0000","last_active":"2024-09-15T23:37:16.056140+0000","last_peered":"2024-09-15T23:37:16.056140+0000","last_clean":"2024-09-15T23:37:16.056140+0000","last_became_active":"2024-09-15T23:37:15.018977+0000","last_became_peered":"2024-09-15T23:37:15.018977+0000","last_unstale":"2024-09-15T23:37:16.056140+0000","last_undegraded":"2024-09-15T23:37:16.056140+0000","last_fullsized":"2024-09-15T23:37:16.056140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:02.654673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013153+0000","last_change":"2024-09-15T23:37:15.013153+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013153+0000","last_undegraded":"2024-09-15T23:37:15.013153+0000","last_fullsized":"2024-09-15T23:37:15.013153+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.027944+0000","last_change":"2024-09-15T23:37:15.024812+0000","last_active":"2024-09-15T23:37:15.027944+0000","last_peered":"2024-09-15T23:37:15.027944+0000","last_clean":"2024-09-15T23:37:15.027944+0000","last_became_active":"2024-09-15T23:37:15.024533+0000","last_became_peered":"2024-09-15T23:37:15.024533+0000","last_unstale":"2024-09-15T23:37:15.027944+0000","last_undegraded":"2024-09-15T23:37:15.027944+0000","last_fullsized":"2024-09-15T23:37:15.027944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:56.378313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046817+0000","last_change":"2024-09-15T23:37:16.043209+0000","last_active":"2024-09-15T23:37:16.046817+0000","last_peered":"2024-09-15T23:37:16.046817+0000","last_clean":"2024-09-15T23:37:16.046817+0000","last_became_active":"2024-09-15T23:37:16.042016+0000","last_became_peered":"2024-09-15T23:37:16.042016+0000","last_unstale":"2024-09-15T23:37:16.046817+0000","last_undegraded":"2024-09-15T23:37:16.046817+0000","last_fullsized":"2024-09-15T23:37:16.046817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:17.167815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008364+0000","last_change":"2024-09-15T23:37:14.008364+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008364+0000","last_undegraded":"2024-09-15T23:37:14.008364+0000","last_fullsized":"2024-09-15T23:37:14.008364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014181+0000","last_change":"2024-09-15T23:37:15.014181+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014181+0000","last_undegraded":"2024-09-15T23:37:15.014181+0000","last_fullsized":"2024-09-15T23:37:15.014181+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056405+0000","last_change":"2024-09-15T23:37:15.023109+0000","last_active":"2024-09-15T23:37:16.056405+0000","last_peered":"2024-09-15T23:37:16.056405+0000","last_clean":"2024-09-15T23:37:16.056405+0000","last_became_active":"2024-09-15T23:37:15.022494+0000","last_became_peered":"2024-09-15T23:37:15.022494+0000","last_unstale":"2024-09-15T23:37:16.056405+0000","last_undegraded":"2024-09-15T23:37:16.056405+0000","last_fullsized":"2024-09-15T23:37:16.056405+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:29.332641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.055917+0000","last_change":"2024-09-15T23:37:15.025209+0000","last_active":"2024-09-15T23:37:16.055917+0000","last_peered":"2024-09-15T23:37:16.055917+0000","last_clean":"2024-09-15T23:37:16.055917+0000","last_became_active":"2024-09-15T23:37:15.022678+0000","last_became_peered":"2024-09-15T23:37:15.022678+0000","last_unstale":"2024-09-15T23:37:16.055917+0000","last_undegraded":"2024-09-15T23:37:16.055917+0000","last_fullsized":"2024-09-15T23:37:16.055917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:01.000785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033653+0000","last_change":"2024-09-15T23:37:16.031990+0000","last_active":"2024-09-15T23:37:16.033653+0000","last_peered":"2024-09-15T23:37:16.033653+0000","last_clean":"2024-09-15T23:37:16.033653+0000","last_became_active":"2024-09-15T23:37:16.031688+0000","last_became_peered":"2024-09-15T23:37:16.031688+0000","last_unstale":"2024-09-15T23:37:16.033653+0000","last_undegraded":"2024-09-15T23:37:16.033653+0000","last_fullsized":"2024-09-15T23:37:16.033653+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:36.654534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052263+0000","last_change":"2024-09-15T23:37:15.029001+0000","last_active":"2024-09-15T23:37:16.052263+0000","last_peered":"2024-09-15T23:37:16.052263+0000","last_clean":"2024-09-15T23:37:16.052263+0000","last_became_active":"2024-09-15T23:37:15.028669+0000","last_became_peered":"2024-09-15T23:37:15.028669+0000","last_unstale":"2024-09-15T23:37:16.052263+0000","last_undegraded":"2024-09-15T23:37:16.052263+0000","last_fullsized":"2024-09-15T23:37:16.052263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:32.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035709+0000","last_change":"2024-09-15T23:37:16.034125+0000","last_active":"2024-09-15T23:37:16.035709+0000","last_peered":"2024-09-15T23:37:16.035709+0000","last_clean":"2024-09-15T23:37:16.035709+0000","last_became_active":"2024-09-15T23:37:16.033801+0000","last_became_peered":"2024-09-15T23:37:16.033801+0000","last_unstale":"2024-09-15T23:37:16.035709+0000","last_undegraded":"2024-09-15T23:37:16.035709+0000","last_fullsized":"2024-09-15T23:37:16.035709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:04.589184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008354+0000","last_change":"2024-09-15T23:37:14.008354+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008354+0000","last_undegraded":"2024-09-15T23:37:14.008354+0000","last_fullsized":"2024-09-15T23:37:14.008354+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013255+0000","last_change":"2024-09-15T23:37:15.013255+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013255+0000","last_undegraded":"2024-09-15T23:37:15.013255+0000","last_fullsized":"2024-09-15T23:37:15.013255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022594+0000","last_change":"2024-09-15T23:37:15.026937+0000","last_active":"2024-09-15T23:37:16.022594+0000","last_peered":"2024-09-15T23:37:16.022594+0000","last_clean":"2024-09-15T23:37:16.022594+0000","last_became_active":"2024-09-15T23:37:15.026293+0000","last_became_peered":"2024-09-15T23:37:15.026293+0000","last_unstale":"2024-09-15T23:37:16.022594+0000","last_undegraded":"2024-09-15T23:37:16.022594+0000","last_fullsized":"2024-09-15T23:37:16.022594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:29.524303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008569+0000","last_change":"2024-09-15T23:37:14.008569+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008569+0000","last_undegraded":"2024-09-15T23:37:14.008569+0000","last_fullsized":"2024-09-15T23:37:14.008569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.045715+0000","last_change":"2024-09-15T23:37:16.044875+0000","last_active":"2024-09-15T23:37:16.045715+0000","last_peered":"2024-09-15T23:37:16.045715+0000","last_clean":"2024-09-15T23:37:16.045715+0000","last_became_active":"2024-09-15T23:37:16.044600+0000","last_became_peered":"2024-09-15T23:37:16.044600+0000","last_unstale":"2024-09-15T23:37:16.045715+0000","last_undegraded":"2024-09-15T23:37:16.045715+0000","last_fullsized":"2024-09-15T23:37:16.045715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:32.153327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020234+0000","last_change":"2024-09-15T23:37:15.025268+0000","last_active":"2024-09-15T23:37:16.020234+0000","last_peered":"2024-09-15T23:37:16.020234+0000","last_clean":"2024-09-15T23:37:16.020234+0000","last_became_active":"2024-09-15T23:37:15.024965+0000","last_became_peered":"2024-09-15T23:37:15.024965+0000","last_unstale":"2024-09-15T23:37:16.020234+0000","last_undegraded":"2024-09-15T23:37:16.020234+0000","last_fullsized":"2024-09-15T23:37:16.020234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:41.312641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.053070+0000","last_change":"2024-09-15T23:37:15.025534+0000","last_active":"2024-09-15T23:37:16.053070+0000","last_peered":"2024-09-15T23:37:16.053070+0000","last_clean":"2024-09-15T23:37:16.053070+0000","last_became_active":"2024-09-15T23:37:15.023762+0000","last_became_peered":"2024-09-15T23:37:15.023762+0000","last_unstale":"2024-09-15T23:37:16.053070+0000","last_undegraded":"2024-09-15T23:37:16.053070+0000","last_fullsized":"2024-09-15T23:37:16.053070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:11.328653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051853+0000","last_change":"2024-09-15T23:37:16.043056+0000","last_active":"2024-09-15T23:37:16.051853+0000","last_peered":"2024-09-15T23:37:16.051853+0000","last_clean":"2024-09-15T23:37:16.051853+0000","last_became_active":"2024-09-15T23:37:16.042639+0000","last_became_peered":"2024-09-15T23:37:16.042639+0000","last_unstale":"2024-09-15T23:37:16.051853+0000","last_undegraded":"2024-09-15T23:37:16.051853+0000","last_fullsized":"2024-09-15T23:37:16.051853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:51.445196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008657+0000","last_change":"2024-09-15T23:37:14.008657+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008657+0000","last_undegraded":"2024-09-15T23:37:14.008657+0000","last_fullsized":"2024-09-15T23:37:14.008657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.009242+0000","last_change":"2024-09-15T23:37:14.009242+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.009242+0000","last_undegraded":"2024-09-15T23:37:14.009242+0000","last_fullsized":"2024-09-15T23:37:14.009242+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.015544+0000","last_change":"2024-09-15T23:37:15.015544+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.015544+0000","last_undegraded":"2024-09-15T23:37:15.015544+0000","last_fullsized":"2024-09-15T23:37:15.015544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.023511+0000","last_change":"2024-09-15T23:37:15.021306+0000","last_active":"2024-09-15T23:37:15.023511+0000","last_peered":"2024-09-15T23:37:15.023511+0000","last_clean":"2024-09-15T23:37:15.023511+0000","last_became_active":"2024-09-15T23:37:15.020973+0000","last_became_peered":"2024-09-15T23:37:15.020973+0000","last_unstale":"2024-09-15T23:37:15.023511+0000","last_undegraded":"2024-09-15T23:37:15.023511+0000","last_fullsized":"2024-09-15T23:37:15.023511+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:09.094953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.034232+0000","last_change":"2024-09-15T23:37:16.028516+0000","last_active":"2024-09-15T23:37:16.034232+0000","last_peered":"2024-09-15T23:37:16.034232+0000","last_clean":"2024-09-15T23:37:16.034232+0000","last_became_active":"2024-09-15T23:37:16.028203+0000","last_became_peered":"2024-09-15T23:37:16.028203+0000","last_unstale":"2024-09-15T23:37:16.034232+0000","last_undegraded":"2024-09-15T23:37:16.034232+0000","last_fullsized":"2024-09-15T23:37:16.034232+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020739+0000","last_change":"2024-09-15T23:37:15.020887+0000","last_active":"2024-09-15T23:37:16.020739+0000","last_peered":"2024-09-15T23:37:16.020739+0000","last_clean":"2024-09-15T23:37:16.020739+0000","last_became_active":"2024-09-15T23:37:15.020611+0000","last_became_peered":"2024-09-15T23:37:15.020611+0000","last_unstale":"2024-09-15T23:37:16.020739+0000","last_undegraded":"2024-09-15T23:37:16.020739+0000","last_fullsized":"2024-09-15T23:37:16.020739+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:55.272773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013244+0000","last_change":"2024-09-15T23:37:15.013244+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013244+0000","last_undegraded":"2024-09-15T23:37:15.013244+0000","last_fullsized":"2024-09-15T23:37:15.013244+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.058124+0000","last_change":"2024-09-15T23:37:15.018587+0000","last_active":"2024-09-15T23:37:16.058124+0000","last_peered":"2024-09-15T23:37:16.058124+0000","last_clean":"2024-09-15T23:37:16.058124+0000","last_became_active":"2024-09-15T23:37:15.018296+0000","last_became_peered":"2024-09-15T23:37:15.018296+0000","last_unstale":"2024-09-15T23:37:16.058124+0000","last_undegraded":"2024-09-15T23:37:16.058124+0000","last_fullsized":"2024-09-15T23:37:16.058124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:01.784529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013616+0000","last_change":"2024-09-15T23:37:15.013616+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013616+0000","last_undegraded":"2024-09-15T23:37:15.013616+0000","last_fullsized":"2024-09-15T23:37:15.013616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052687+0000","last_change":"2024-09-15T23:37:15.025903+0000","last_active":"2024-09-15T23:37:16.052687+0000","last_peered":"2024-09-15T23:37:16.052687+0000","last_clean":"2024-09-15T23:37:16.052687+0000","last_became_active":"2024-09-15T23:37:15.025212+0000","last_became_peered":"2024-09-15T23:37:15.025212+0000","last_unstale":"2024-09-15T23:37:16.052687+0000","last_undegraded":"2024-09-15T23:37:16.052687+0000","last_fullsized":"2024-09-15T23:37:16.052687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:41.537186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047929+0000","last_change":"2024-09-15T23:37:16.042373+0000","last_active":"2024-09-15T23:37:16.047929+0000","last_peered":"2024-09-15T23:37:16.047929+0000","last_clean":"2024-09-15T23:37:16.047929+0000","last_became_active":"2024-09-15T23:37:16.040111+0000","last_became_peered":"2024-09-15T23:37:16.040111+0000","last_unstale":"2024-09-15T23:37:16.047929+0000","last_undegraded":"2024-09-15T23:37:16.047929+0000","last_fullsized":"2024-09-15T23:37:16.047929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:36:14.588537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052462+0000","last_change":"2024-09-15T23:37:15.024606+0000","last_active":"2024-09-15T23:37:16.052462+0000","last_peered":"2024-09-15T23:37:16.052462+0000","last_clean":"2024-09-15T23:37:16.052462+0000","last_became_active":"2024-09-15T23:37:15.024349+0000","last_became_peered":"2024-09-15T23:37:15.024349+0000","last_unstale":"2024-09-15T23:37:16.052462+0000","last_undegraded":"2024-09-15T23:37:16.052462+0000","last_fullsized":"2024-09-15T23:37:16.052462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:23.692350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047894+0000","last_change":"2024-09-15T23:37:16.041862+0000","last_active":"2024-09-15T23:37:16.047894+0000","last_peered":"2024-09-15T23:37:16.047894+0000","last_clean":"2024-09-15T23:37:16.047894+0000","last_became_active":"2024-09-15T23:37:16.041522+0000","last_became_peered":"2024-09-15T23:37:16.041522+0000","last_unstale":"2024-09-15T23:37:16.047894+0000","last_undegraded":"2024-09-15T23:37:16.047894+0000","last_fullsized":"2024-09-15T23:37:16.047894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:08.290613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.025129+0000","last_change":"2024-09-15T23:37:15.023432+0000","last_active":"2024-09-15T23:37:15.025129+0000","last_peered":"2024-09-15T23:37:15.025129+0000","last_clean":"2024-09-15T23:37:15.025129+0000","last_became_active":"2024-09-15T23:37:15.023142+0000","last_became_peered":"2024-09-15T23:37:15.023142+0000","last_unstale":"2024-09-15T23:37:15.025129+0000","last_undegraded":"2024-09-15T23:37:15.025129+0000","last_fullsized":"2024-09-15T23:37:15.025129+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:41.537186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.011492+0000","last_change":"2024-09-15T23:37:14.011492+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.011492+0000","last_undegraded":"2024-09-15T23:37:14.011492+0000","last_fullsized":"2024-09-15T23:37:14.011492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.025069+0000","last_change":"2024-09-15T23:37:16.023795+0000","last_active":"2024-09-15T23:37:16.025069+0000","last_peered":"2024-09-15T23:37:16.025069+0000","last_clean":"2024-09-15T23:37:16.025069+0000","last_became_active":"2024-09-15T23:37:16.023493+0000","last_became_peered":"2024-09-15T23:37:16.023493+0000","last_unstale":"2024-09-15T23:37:16.025069+0000","last_undegraded":"2024-09-15T23:37:16.025069+0000","last_fullsized":"2024-09-15T23:37:16.025069+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:19.252190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.011446+0000","last_change":"2024-09-15T23:37:14.011446+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.011446+0000","last_undegraded":"2024-09-15T23:37:14.011446+0000","last_fullsized":"2024-09-15T23:37:14.011446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.022413+0000","last_change":"2024-09-15T23:37:15.020340+0000","last_active":"2024-09-15T23:37:15.022413+0000","last_peered":"2024-09-15T23:37:15.022413+0000","last_clean":"2024-09-15T23:37:15.022413+0000","last_became_active":"2024-09-15T23:37:15.020031+0000","last_became_peered":"2024-09-15T23:37:15.020031+0000","last_unstale":"2024-09-15T23:37:15.022413+0000","last_undegraded":"2024-09-15T23:37:15.022413+0000","last_fullsized":"2024-09-15T23:37:15.022413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:50.440261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051420+0000","last_change":"2024-09-15T23:37:16.049420+0000","last_active":"2024-09-15T23:37:16.051420+0000","last_peered":"2024-09-15T23:37:16.051420+0000","last_clean":"2024-09-15T23:37:16.051420+0000","last_became_active":"2024-09-15T23:37:16.049125+0000","last_became_peered":"2024-09-15T23:37:16.049125+0000","last_unstale":"2024-09-15T23:37:16.051420+0000","last_undegraded":"2024-09-15T23:37:16.051420+0000","last_fullsized":"2024-09-15T23:37:16.051420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:56.834193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.024060+0000","last_change":"2024-09-15T23:37:15.030423+0000","last_active":"2024-09-15T23:37:16.024060+0000","last_peered":"2024-09-15T23:37:16.024060+0000","last_clean":"2024-09-15T23:37:16.024060+0000","last_became_active":"2024-09-15T23:37:15.030077+0000","last_became_peered":"2024-09-15T23:37:15.030077+0000","last_unstale":"2024-09-15T23:37:16.024060+0000","last_undegraded":"2024-09-15T23:37:16.024060+0000","last_fullsized":"2024-09-15T23:37:16.024060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.262938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.021265+0000","last_change":"2024-09-15T23:37:15.018265+0000","last_active":"2024-09-15T23:37:15.021265+0000","last_peered":"2024-09-15T23:37:15.021265+0000","last_clean":"2024-09-15T23:37:15.021265+0000","last_became_active":"2024-09-15T23:37:15.017961+0000","last_became_peered":"2024-09-15T23:37:15.017961+0000","last_unstale":"2024-09-15T23:37:15.021265+0000","last_undegraded":"2024-09-15T23:37:15.021265+0000","last_fullsized":"2024-09-15T23:37:15.021265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:36:13.583601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.014174+0000","last_change":"2024-09-15T23:37:15.014174+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.014174+0000","last_undegraded":"2024-09-15T23:37:15.014174+0000","last_fullsized":"2024-09-15T23:37:15.014174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052977+0000","last_change":"2024-09-15T23:37:15.021730+0000","last_active":"2024-09-15T23:37:16.052977+0000","last_peered":"2024-09-15T23:37:16.052977+0000","last_clean":"2024-09-15T23:37:16.052977+0000","last_became_active":"2024-09-15T23:37:15.021217+0000","last_became_peered":"2024-09-15T23:37:15.021217+0000","last_unstale":"2024-09-15T23:37:16.052977+0000","last_undegraded":"2024-09-15T23:37:16.052977+0000","last_fullsized":"2024-09-15T23:37:16.052977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:43:54.440674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052881+0000","last_change":"2024-09-15T23:37:16.050557+0000","last_active":"2024-09-15T23:37:16.052881+0000","last_peered":"2024-09-15T23:37:16.052881+0000","last_clean":"2024-09-15T23:37:16.052881+0000","last_became_active":"2024-09-15T23:37:16.050267+0000","last_became_peered":"2024-09-15T23:37:16.050267+0000","last_unstale":"2024-09-15T23:37:16.052881+0000","last_undegraded":"2024-09-15T23:37:16.052881+0000","last_fullsized":"2024-09-15T23:37:16.052881+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:01.512554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.030092+0000","last_change":"2024-09-15T23:37:15.029187+0000","last_active":"2024-09-15T23:37:15.030092+0000","last_peered":"2024-09-15T23:37:15.030092+0000","last_clean":"2024-09-15T23:37:15.030092+0000","last_became_active":"2024-09-15T23:37:15.028731+0000","last_became_peered":"2024-09-15T23:37:15.028731+0000","last_unstale":"2024-09-15T23:37:15.030092+0000","last_undegraded":"2024-09-15T23:37:15.030092+0000","last_fullsized":"2024-09-15T23:37:15.030092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.262938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T23:37:15.003519+0000","last_change":"2024-09-15T23:37:15.003519+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.003519+0000","last_undegraded":"2024-09-15T23:37:15.003519+0000","last_fullsized":"2024-09-15T23:37:15.003519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T23:37:15.029676+0000","last_change":"2024-09-15T23:37:15.028765+0000","last_active":"2024-09-15T23:37:15.029676+0000","last_peered":"2024-09-15T23:37:15.029676+0000","last_clean":"2024-09-15T23:37:15.029676+0000","last_became_active":"2024-09-15T23:37:15.028426+0000","last_became_peered":"2024-09-15T23:37:15.028426+0000","last_unstale":"2024-09-15T23:37:15.029676+0000","last_undegraded":"2024-09-15T23:37:15.029676+0000","last_fullsized":"2024-09-15T23:37:15.029676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:32.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.012677+0000","last_change":"2024-09-15T23:37:15.012677+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.012677+0000","last_undegraded":"2024-09-15T23:37:15.012677+0000","last_fullsized":"2024-09-15T23:37:15.012677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008364+0000","last_change":"2024-09-15T23:37:14.008364+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008364+0000","last_undegraded":"2024-09-15T23:37:14.008364+0000","last_fullsized":"2024-09-15T23:37:14.008364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-15T23:37:15.013767+0000","last_change":"2024-09-15T23:37:15.013767+0000","last_active":"2024-09-15T23:37:15.003519+0000","last_peered":"2024-09-15T23:37:15.003519+0000","last_clean":"2024-09-15T23:37:15.003519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:15.013767+0000","last_undegraded":"2024-09-15T23:37:15.013767+0000","last_fullsized":"2024-09-15T23:37:15.013767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.026104+0000","last_change":"2024-09-15T23:37:15.023867+0000","last_active":"2024-09-15T23:37:16.026104+0000","last_peered":"2024-09-15T23:37:16.026104+0000","last_clean":"2024-09-15T23:37:16.026104+0000","last_became_active":"2024-09-15T23:37:15.023470+0000","last_became_peered":"2024-09-15T23:37:15.023470+0000","last_unstale":"2024-09-15T23:37:16.026104+0000","last_undegraded":"2024-09-15T23:37:16.026104+0000","last_fullsized":"2024-09-15T23:37:16.026104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:13.335910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044234+0000","last_change":"2024-09-15T23:37:16.041472+0000","last_active":"2024-09-15T23:37:16.044234+0000","last_peered":"2024-09-15T23:37:16.044234+0000","last_clean":"2024-09-15T23:37:16.044234+0000","last_became_active":"2024-09-15T23:37:16.040586+0000","last_became_peered":"2024-09-15T23:37:16.040586+0000","last_unstale":"2024-09-15T23:37:16.044234+0000","last_undegraded":"2024-09-15T23:37:16.044234+0000","last_fullsized":"2024-09-15T23:37:16.044234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:01.946016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020562+0000","last_change":"2024-09-15T23:37:15.022433+0000","last_active":"2024-09-15T23:37:16.020562+0000","last_peered":"2024-09-15T23:37:16.020562+0000","last_clean":"2024-09-15T23:37:16.020562+0000","last_became_active":"2024-09-15T23:37:15.022150+0000","last_became_peered":"2024-09-15T23:37:15.022150+0000","last_unstale":"2024-09-15T23:37:16.020562+0000","last_undegraded":"2024-09-15T23:37:16.020562+0000","last_fullsized":"2024-09-15T23:37:16.020562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:06.425003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033731+0000","last_change":"2024-09-15T23:37:16.031076+0000","last_active":"2024-09-15T23:37:16.033731+0000","last_peered":"2024-09-15T23:37:16.033731+0000","last_clean":"2024-09-15T23:37:16.033731+0000","last_became_active":"2024-09-15T23:37:16.030718+0000","last_became_peered":"2024-09-15T23:37:16.030718+0000","last_unstale":"2024-09-15T23:37:16.033731+0000","last_undegraded":"2024-09-15T23:37:16.033731+0000","last_fullsized":"2024-09-15T23:37:16.033731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:31.411462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-15T23:37:14.008468+0000","last_change":"2024-09-15T23:37:14.008468+0000","last_active":"2024-09-15T23:37:13.998584+0000","last_peered":"2024-09-15T23:37:13.998584+0000","last_clean":"2024-09-15T23:37:13.998584+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T23:37:14.008468+0000","last_undegraded":"2024-09-15T23:37:14.008468+0000","last_fullsized":"2024-09-15T23:37:14.008468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035661+0000","last_change":"2024-09-15T23:37:16.034326+0000","last_active":"2024-09-15T23:37:16.035661+0000","last_peered":"2024-09-15T23:37:16.035661+0000","last_clean":"2024-09-15T23:37:16.035661+0000","last_became_active":"2024-09-15T23:37:16.033729+0000","last_became_peered":"2024-09-15T23:37:16.033729+0000","last_unstale":"2024-09-15T23:37:16.035661+0000","last_undegraded":"2024-09-15T23:37:16.035661+0000","last_fullsized":"2024-09-15T23:37:16.035661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:50.331476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.027426+0000","last_change":"2024-09-15T23:37:15.024501+0000","last_active":"2024-09-15T23:37:16.027426+0000","last_peered":"2024-09-15T23:37:16.027426+0000","last_clean":"2024-09-15T23:37:16.027426+0000","last_became_active":"2024-09-15T23:37:15.024195+0000","last_became_peered":"2024-09-15T23:37:15.024195+0000","last_unstale":"2024-09-15T23:37:16.027426+0000","last_undegraded":"2024-09-15T23:37:16.027426+0000","last_fullsized":"2024-09-15T23:37:16.027426+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:31.148391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022665+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:37:16.022665+0000","last_peered":"2024-09-15T23:37:16.022665+0000","last_clean":"2024-09-15T23:37:16.022665+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:16.022665+0000","last_undegraded":"2024-09-15T23:37:16.022665+0000","last_fullsized":"2024-09-15T23:37:16.022665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":185007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":446464,"data_stored":187201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1028096,"data_stored":775397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":438272,"data_stored":185099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":417792,"data_stored":177840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":417792,"data_stored":177840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":761541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":171173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:37:20.998 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-09-15T23:37:20.997+0000 7f9413af7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T23:37:23.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:37:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.542+0000 7f60d6682640 1 Processor -- start 2024-09-15T23:37:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.542+0000 7f60d6682640 1 -- start start 2024-09-15T23:37:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60d6682640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60d00d0f90 0x7f60d00d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60d6682640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d00d0a30 con 0x7f60d00d52e0 2024-09-15T23:37:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60d6682640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d006dd80 con 0x7f60d015e840 2024-09-15T23:37:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60d6682640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f60d00cfe90 con 0x7f60d00d0f90 2024-09-15T23:37:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60d4c1f640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60d00d0f90 0x7f60d00d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60cf7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f60d00d52e0 0x7f60d006d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51956/0 (socket says 172.21.15.7:51956) 2024-09-15T23:37:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60cf7fe640 1 -- 172.21.15.7:0/2163416973 learned_addr learned my addr 172.21.15.7:0/2163416973 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60d4c1f640 1 -- 172.21.15.7:0/2163416973 >> v1:172.21.15.18:6789/0 conn(0x7f60d015e840 legacy=0x7f60d00d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60d4c1f640 1 -- 172.21.15.7:0/2163416973 >> v1:172.21.15.7:6789/0 conn(0x7f60d00d52e0 legacy=0x7f60d006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60ceffd640 1 -- 172.21.15.7:0/2163416973 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15490444 0 0) 0x7f60d00d0a30 con 0x7f60d00d52e0 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.543+0000 7f60d4c1f640 1 -- 172.21.15.7:0/2163416973 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d00cf560 con 0x7f60d00d0f90 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.544+0000 7f60d4c1f640 1 --2- 172.21.15.7:0/2163416973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d0f90 0x7f60d00d13c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f60d00d0010 tx=0x7f60c802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfd84da5c44f885 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.544+0000 7f60ceffd640 1 -- 172.21.15.7:0/2163416973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8007740 con 0x7f60d00d0f90 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.544+0000 7f60ceffd640 1 -- 172.21.15.7:0/2163416973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c80078e0 con 0x7f60d00d0f90 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.544+0000 7f60ceffd640 1 -- 172.21.15.7:0/2163416973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8007cb0 con 0x7f60d00d0f90 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.545+0000 7f60d6682640 1 -- 172.21.15.7:0/2163416973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d0f90 msgr2=0x7f60d00d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.545+0000 7f60d6682640 1 --2- 172.21.15.7:0/2163416973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d0f90 0x7f60d00d13c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f60d00d0010 tx=0x7f60c802a6f0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.545+0000 7f60d6682640 1 -- 172.21.15.7:0/2163416973 shutdown_connections 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.545+0000 7f60d6682640 1 --2- 172.21.15.7:0/2163416973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d0f90 0x7f60d00d13c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.545+0000 7f60d6682640 1 -- 172.21.15.7:0/2163416973 >> 172.21.15.7:0/2163416973 conn(0x7f60d008bfd0 msgr2=0x7f60d00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:23.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.545+0000 7f60d6682640 1 -- 172.21.15.7:0/2163416973 shutdown_connections 2024-09-15T23:37:23.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.545+0000 7f60d6682640 1 -- 172.21.15.7:0/2163416973 wait complete. 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.546+0000 7f60d6682640 1 Processor -- start 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.546+0000 7f60d6682640 1 -- start start 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60d6682640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60d00d0f90 0x7f60d00b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60d6682640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d52e0 0x7f60d01ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60d6682640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f60d015e840 0x7f60d01f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60d6682640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60d00cfb20 con 0x7f60d00d52e0 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60d6682640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f60d00d1900 con 0x7f60d00d0f90 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60d6682640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f60d006e030 con 0x7f60d015e840 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d52e0 0x7f60d01ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d52e0 0x7f60d01ee2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54570/0 (socket says 172.21.15.7:54570) 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cf7fe640 1 -- 172.21.15.7:0/1270856488 learned_addr learned my addr 172.21.15.7:0/1270856488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60d4c1f640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f60d015e840 0x7f60d01f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cffff640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60d00d0f90 0x7f60d00b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cf7fe640 1 -- 172.21.15.7:0/1270856488 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f60d015e840 msgr2=0x7f60d01f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cf7fe640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f60d015e840 0x7f60d01f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cf7fe640 1 -- 172.21.15.7:0/1270856488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60d00d0f90 msgr2=0x7f60d00b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cf7fe640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60d00d0f90 0x7f60d00b6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cf7fe640 1 -- 172.21.15.7:0/1270856488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d00b4c80 con 0x7f60d00d52e0 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.547+0000 7f60cf7fe640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d52e0 0x7f60d01ee2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f60c400c730 tx=0x7f60c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.548+0000 7f60ccff9640 1 -- 172.21.15.7:0/1270856488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4007270 con 0x7f60d00d52e0 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.548+0000 7f60ccff9640 1 -- 172.21.15.7:0/1270856488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c4007430 con 0x7f60d00d52e0 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.548+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d00b4f60 con 0x7f60d00d52e0 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.548+0000 7f60ccff9640 1 -- 172.21.15.7:0/1270856488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40059e0 con 0x7f60d00d52e0 2024-09-15T23:37:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.548+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d00b5520 con 0x7f60d00d52e0 2024-09-15T23:37:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.549+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d015ec20 con 0x7f60d00d52e0 2024-09-15T23:37:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.549+0000 7f60ccff9640 1 -- 172.21.15.7:0/1270856488 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f60c4005b80 con 0x7f60d00d52e0 2024-09-15T23:37:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.550+0000 7f60ccff9640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60bc06c2e0 0x7f60bc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.550+0000 7f60ccff9640 1 -- 172.21.15.7:0/1270856488 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f60c408b120 con 0x7f60d00d52e0 2024-09-15T23:37:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.550+0000 7f60cffff640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60bc06c2e0 0x7f60bc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.550+0000 7f60cffff640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60bc06c2e0 0x7f60bc06e7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60d00d3fd0 tx=0x7f60c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.554+0000 7f60ccff9640 1 -- 172.21.15.7:0/1270856488 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f60c4001d00 con 0x7f60d00d52e0 2024-09-15T23:37:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.727+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60d00d1cb0 con 0x7f60bc06c2e0 2024-09-15T23:37:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.733+0000 7f60ccff9640 1 -- 172.21.15.7:0/1270856488 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251922 (secure 0 0 0) 0x7f60d00d1cb0 con 0x7f60bc06c2e0 2024-09-15T23:37:23.736 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:23.737 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60bc06c2e0 msgr2=0x7f60bc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60bc06c2e0 0x7f60bc06e7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60d00d3fd0 tx=0x7f60c0009040 comp rx=0 tx=0).stop 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d52e0 msgr2=0x7f60d01ee2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d52e0 0x7f60d01ee2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f60c400c730 tx=0x7f60c400cbf0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 shutdown_connections 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f60bc06c2e0 0x7f60bc06e7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f60d015e840 0x7f60d01f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d00d52e0 0x7f60d01ee2b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 --2- 172.21.15.7:0/1270856488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60d00d0f90 0x7f60d00b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 >> 172.21.15.7:0/1270856488 conn(0x7f60d008bfd0 msgr2=0x7f60d00b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 shutdown_connections 2024-09-15T23:37:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.736+0000 7f60d6682640 1 -- 172.21.15.7:0/1270856488 wait complete. 2024-09-15T23:37:23.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-15T23:37:21.823713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254164,"kb_used_data":4628,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720556,"statfs":{"total":773094113280,"available":772833849344,"internally_reserved":0,"allocated":4739072,"data_stored":2692290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.816576"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.032264+0000","last_change":"2024-09-15T23:37:16.036053+0000","last_active":"2024-09-15T23:37:17.032264+0000","last_peered":"2024-09-15T23:37:17.032264+0000","last_clean":"2024-09-15T23:37:17.032264+0000","last_became_active":"2024-09-15T23:37:16.035759+0000","last_became_peered":"2024-09-15T23:37:16.035759+0000","last_unstale":"2024-09-15T23:37:17.032264+0000","last_undegraded":"2024-09-15T23:37:17.032264+0000","last_fullsized":"2024-09-15T23:37:17.032264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:49.540617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031220+0000","last_change":"2024-09-15T23:37:16.023909+0000","last_active":"2024-09-15T23:37:17.031220+0000","last_peered":"2024-09-15T23:37:17.031220+0000","last_clean":"2024-09-15T23:37:17.031220+0000","last_became_active":"2024-09-15T23:37:16.023628+0000","last_became_peered":"2024-09-15T23:37:16.023628+0000","last_unstale":"2024-09-15T23:37:17.031220+0000","last_undegraded":"2024-09-15T23:37:17.031220+0000","last_fullsized":"2024-09-15T23:37:17.031220+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:24.372796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031895+0000","last_change":"2024-09-15T23:37:16.043513+0000","last_active":"2024-09-15T23:37:17.031895+0000","last_peered":"2024-09-15T23:37:17.031895+0000","last_clean":"2024-09-15T23:37:17.031895+0000","last_became_active":"2024-09-15T23:37:16.043218+0000","last_became_peered":"2024-09-15T23:37:16.043218+0000","last_unstale":"2024-09-15T23:37:17.031895+0000","last_undegraded":"2024-09-15T23:37:17.031895+0000","last_fullsized":"2024-09-15T23:37:17.031895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:29.303162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031749+0000","last_change":"2024-09-15T23:37:16.037969+0000","last_active":"2024-09-15T23:37:17.031749+0000","last_peered":"2024-09-15T23:37:17.031749+0000","last_clean":"2024-09-15T23:37:17.031749+0000","last_became_active":"2024-09-15T23:37:16.037683+0000","last_became_peered":"2024-09-15T23:37:16.037683+0000","last_unstale":"2024-09-15T23:37:17.031749+0000","last_undegraded":"2024-09-15T23:37:17.031749+0000","last_fullsized":"2024-09-15T23:37:17.031749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:44.128246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.034483+0000","last_change":"2024-09-15T23:37:16.033841+0000","last_active":"2024-09-15T23:37:17.034483+0000","last_peered":"2024-09-15T23:37:17.034483+0000","last_clean":"2024-09-15T23:37:17.034483+0000","last_became_active":"2024-09-15T23:37:16.031798+0000","last_became_peered":"2024-09-15T23:37:16.031798+0000","last_unstale":"2024-09-15T23:37:17.034483+0000","last_undegraded":"2024-09-15T23:37:17.034483+0000","last_fullsized":"2024-09-15T23:37:17.034483+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:49.089522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030132+0000","last_change":"2024-09-15T23:37:16.029349+0000","last_active":"2024-09-15T23:37:17.030132+0000","last_peered":"2024-09-15T23:37:17.030132+0000","last_clean":"2024-09-15T23:37:17.030132+0000","last_became_active":"2024-09-15T23:37:16.029023+0000","last_became_peered":"2024-09-15T23:37:16.029023+0000","last_unstale":"2024-09-15T23:37:17.030132+0000","last_undegraded":"2024-09-15T23:37:17.030132+0000","last_fullsized":"2024-09-15T23:37:17.030132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:05.128368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048204+0000","last_change":"2024-09-15T23:37:16.039624+0000","last_active":"2024-09-15T23:37:16.048204+0000","last_peered":"2024-09-15T23:37:16.048204+0000","last_clean":"2024-09-15T23:37:16.048204+0000","last_became_active":"2024-09-15T23:37:16.038879+0000","last_became_peered":"2024-09-15T23:37:16.038879+0000","last_unstale":"2024-09-15T23:37:16.048204+0000","last_undegraded":"2024-09-15T23:37:16.048204+0000","last_fullsized":"2024-09-15T23:37:16.048204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:51.545065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030668+0000","last_change":"2024-09-15T23:37:16.049955+0000","last_active":"2024-09-15T23:37:17.030668+0000","last_peered":"2024-09-15T23:37:17.030668+0000","last_clean":"2024-09-15T23:37:17.030668+0000","last_became_active":"2024-09-15T23:37:16.049466+0000","last_became_peered":"2024-09-15T23:37:16.049466+0000","last_unstale":"2024-09-15T23:37:17.030668+0000","last_undegraded":"2024-09-15T23:37:17.030668+0000","last_fullsized":"2024-09-15T23:37:17.030668+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:03.475973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.034769+0000","last_change":"2024-09-15T23:37:16.030958+0000","last_active":"2024-09-15T23:37:17.034769+0000","last_peered":"2024-09-15T23:37:17.034769+0000","last_clean":"2024-09-15T23:37:17.034769+0000","last_became_active":"2024-09-15T23:37:16.027258+0000","last_became_peered":"2024-09-15T23:37:16.027258+0000","last_unstale":"2024-09-15T23:37:17.034769+0000","last_undegraded":"2024-09-15T23:37:17.034769+0000","last_fullsized":"2024-09-15T23:37:17.034769+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:02.789465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051987+0000","last_change":"2024-09-15T23:37:16.043510+0000","last_active":"2024-09-15T23:37:16.051987+0000","last_peered":"2024-09-15T23:37:16.051987+0000","last_clean":"2024-09-15T23:37:16.051987+0000","last_became_active":"2024-09-15T23:37:16.043208+0000","last_became_peered":"2024-09-15T23:37:16.043208+0000","last_unstale":"2024-09-15T23:37:16.051987+0000","last_undegraded":"2024-09-15T23:37:16.051987+0000","last_fullsized":"2024-09-15T23:37:16.051987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:10.099889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052859+0000","last_change":"2024-09-15T23:37:16.050348+0000","last_active":"2024-09-15T23:37:16.052859+0000","last_peered":"2024-09-15T23:37:16.052859+0000","last_clean":"2024-09-15T23:37:16.052859+0000","last_became_active":"2024-09-15T23:37:16.049871+0000","last_became_peered":"2024-09-15T23:37:16.049871+0000","last_unstale":"2024-09-15T23:37:16.052859+0000","last_undegraded":"2024-09-15T23:37:16.052859+0000","last_fullsized":"2024-09-15T23:37:16.052859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:23.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.034617+0000","last_change":"2024-09-15T23:37:16.030608+0000","last_active":"2024-09-15T23:37:17.034617+0000","last_peered":"2024-09-15T23:37:17.034617+0000","last_clean":"2024-09-15T23:37:17.034617+0000","last_became_active":"2024-09-15T23:37:16.030316+0000","last_became_peered":"2024-09-15T23:37:16.030316+0000","last_unstale":"2024-09-15T23:37:17.034617+0000","last_undegraded":"2024-09-15T23:37:17.034617+0000","last_fullsized":"2024-09-15T23:37:17.034617+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:03.659609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047271+0000","last_change":"2024-09-15T23:37:16.037928+0000","last_active":"2024-09-15T23:37:16.047271+0000","last_peered":"2024-09-15T23:37:16.047271+0000","last_clean":"2024-09-15T23:37:16.047271+0000","last_became_active":"2024-09-15T23:37:16.037645+0000","last_became_peered":"2024-09-15T23:37:16.037645+0000","last_unstale":"2024-09-15T23:37:16.047271+0000","last_undegraded":"2024-09-15T23:37:16.047271+0000","last_fullsized":"2024-09-15T23:37:16.047271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:29.660110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048427+0000","last_change":"2024-09-15T23:37:16.046081+0000","last_active":"2024-09-15T23:37:16.048427+0000","last_peered":"2024-09-15T23:37:16.048427+0000","last_clean":"2024-09-15T23:37:16.048427+0000","last_became_active":"2024-09-15T23:37:16.045733+0000","last_became_peered":"2024-09-15T23:37:16.045733+0000","last_unstale":"2024-09-15T23:37:16.048427+0000","last_undegraded":"2024-09-15T23:37:16.048427+0000","last_fullsized":"2024-09-15T23:37:16.048427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036156+0000","last_change":"2024-09-15T23:37:16.024098+0000","last_active":"2024-09-15T23:37:16.036156+0000","last_peered":"2024-09-15T23:37:16.036156+0000","last_clean":"2024-09-15T23:37:16.036156+0000","last_became_active":"2024-09-15T23:37:16.023753+0000","last_became_peered":"2024-09-15T23:37:16.023753+0000","last_unstale":"2024-09-15T23:37:16.036156+0000","last_undegraded":"2024-09-15T23:37:16.036156+0000","last_fullsized":"2024-09-15T23:37:16.036156+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:34.586227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052091+0000","last_change":"2024-09-15T23:37:16.048043+0000","last_active":"2024-09-15T23:37:16.052091+0000","last_peered":"2024-09-15T23:37:16.052091+0000","last_clean":"2024-09-15T23:37:16.052091+0000","last_became_active":"2024-09-15T23:37:16.047587+0000","last_became_peered":"2024-09-15T23:37:16.047587+0000","last_unstale":"2024-09-15T23:37:16.052091+0000","last_undegraded":"2024-09-15T23:37:16.052091+0000","last_fullsized":"2024-09-15T23:37:16.052091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:57.383248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035550+0000","last_change":"2024-09-15T23:37:16.034026+0000","last_active":"2024-09-15T23:37:16.035550+0000","last_peered":"2024-09-15T23:37:16.035550+0000","last_clean":"2024-09-15T23:37:16.035550+0000","last_became_active":"2024-09-15T23:37:16.033385+0000","last_became_peered":"2024-09-15T23:37:16.033385+0000","last_unstale":"2024-09-15T23:37:16.035550+0000","last_undegraded":"2024-09-15T23:37:16.035550+0000","last_fullsized":"2024-09-15T23:37:16.035550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:33.923553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035623+0000","last_change":"2024-09-15T23:37:16.034152+0000","last_active":"2024-09-15T23:37:16.035623+0000","last_peered":"2024-09-15T23:37:16.035623+0000","last_clean":"2024-09-15T23:37:16.035623+0000","last_became_active":"2024-09-15T23:37:16.029607+0000","last_became_peered":"2024-09-15T23:37:16.029607+0000","last_unstale":"2024-09-15T23:37:16.035623+0000","last_undegraded":"2024-09-15T23:37:16.035623+0000","last_fullsized":"2024-09-15T23:37:16.035623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:55.516820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.897039+0000","last_change":"2024-09-15T23:37:16.039828+0000","last_active":"2024-09-15T23:37:17.897039+0000","last_peered":"2024-09-15T23:37:17.897039+0000","last_clean":"2024-09-15T23:37:17.897039+0000","last_became_active":"2024-09-15T23:37:16.039537+0000","last_became_peered":"2024-09-15T23:37:16.039537+0000","last_unstale":"2024-09-15T23:37:17.897039+0000","last_undegraded":"2024-09-15T23:37:17.897039+0000","last_fullsized":"2024-09-15T23:37:17.897039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:51.750497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042577+0000","last_change":"2024-09-15T23:37:16.037756+0000","last_active":"2024-09-15T23:37:16.042577+0000","last_peered":"2024-09-15T23:37:16.042577+0000","last_clean":"2024-09-15T23:37:16.042577+0000","last_became_active":"2024-09-15T23:37:16.037309+0000","last_became_peered":"2024-09-15T23:37:16.037309+0000","last_unstale":"2024-09-15T23:37:16.042577+0000","last_undegraded":"2024-09-15T23:37:16.042577+0000","last_fullsized":"2024-09-15T23:37:16.042577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:45.986835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.042641+0000","last_change":"2024-09-15T23:37:16.037039+0000","last_active":"2024-09-15T23:37:16.042641+0000","last_peered":"2024-09-15T23:37:16.042641+0000","last_clean":"2024-09-15T23:37:16.042641+0000","last_became_active":"2024-09-15T23:37:16.036774+0000","last_became_peered":"2024-09-15T23:37:16.036774+0000","last_unstale":"2024-09-15T23:37:16.042641+0000","last_undegraded":"2024-09-15T23:37:16.042641+0000","last_fullsized":"2024-09-15T23:37:16.042641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:30.804394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.040576+0000","last_change":"2024-09-15T23:37:16.034353+0000","last_active":"2024-09-15T23:37:16.040576+0000","last_peered":"2024-09-15T23:37:16.040576+0000","last_clean":"2024-09-15T23:37:16.040576+0000","last_became_active":"2024-09-15T23:37:16.033979+0000","last_became_peered":"2024-09-15T23:37:16.033979+0000","last_unstale":"2024-09-15T23:37:16.040576+0000","last_undegraded":"2024-09-15T23:37:16.040576+0000","last_fullsized":"2024-09-15T23:37:16.040576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:05.128368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.036134+0000","last_change":"2024-09-15T23:37:16.027056+0000","last_active":"2024-09-15T23:37:16.036134+0000","last_peered":"2024-09-15T23:37:16.036134+0000","last_clean":"2024-09-15T23:37:16.036134+0000","last_became_active":"2024-09-15T23:37:16.026780+0000","last_became_peered":"2024-09-15T23:37:16.026780+0000","last_unstale":"2024-09-15T23:37:16.036134+0000","last_undegraded":"2024-09-15T23:37:16.036134+0000","last_fullsized":"2024-09-15T23:37:16.036134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:43.279890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030828+0000","last_change":"2024-09-15T23:37:16.042765+0000","last_active":"2024-09-15T23:37:17.030828+0000","last_peered":"2024-09-15T23:37:17.030828+0000","last_clean":"2024-09-15T23:37:17.030828+0000","last_became_active":"2024-09-15T23:37:16.042483+0000","last_became_peered":"2024-09-15T23:37:16.042483+0000","last_unstale":"2024-09-15T23:37:17.030828+0000","last_undegraded":"2024-09-15T23:37:17.030828+0000","last_fullsized":"2024-09-15T23:37:17.030828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:36.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044498+0000","last_change":"2024-09-15T23:37:16.042298+0000","last_active":"2024-09-15T23:37:16.044498+0000","last_peered":"2024-09-15T23:37:16.044498+0000","last_clean":"2024-09-15T23:37:16.044498+0000","last_became_active":"2024-09-15T23:37:16.041961+0000","last_became_peered":"2024-09-15T23:37:16.041961+0000","last_unstale":"2024-09-15T23:37:16.044498+0000","last_undegraded":"2024-09-15T23:37:16.044498+0000","last_fullsized":"2024-09-15T23:37:16.044498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:14.340846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.032168+0000","last_change":"2024-09-15T23:37:16.042516+0000","last_active":"2024-09-15T23:37:17.032168+0000","last_peered":"2024-09-15T23:37:17.032168+0000","last_clean":"2024-09-15T23:37:17.032168+0000","last_became_active":"2024-09-15T23:37:16.042188+0000","last_became_peered":"2024-09-15T23:37:16.042188+0000","last_unstale":"2024-09-15T23:37:17.032168+0000","last_undegraded":"2024-09-15T23:37:17.032168+0000","last_fullsized":"2024-09-15T23:37:17.032168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:50.387429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033699+0000","last_change":"2024-09-15T23:37:16.031292+0000","last_active":"2024-09-15T23:37:16.033699+0000","last_peered":"2024-09-15T23:37:16.033699+0000","last_clean":"2024-09-15T23:37:16.033699+0000","last_became_active":"2024-09-15T23:37:16.030975+0000","last_became_peered":"2024-09-15T23:37:16.030975+0000","last_unstale":"2024-09-15T23:37:16.033699+0000","last_undegraded":"2024-09-15T23:37:16.033699+0000","last_fullsized":"2024-09-15T23:37:16.033699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:12.360351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048310+0000","last_change":"2024-09-15T23:37:16.039927+0000","last_active":"2024-09-15T23:37:16.048310+0000","last_peered":"2024-09-15T23:37:16.048310+0000","last_clean":"2024-09-15T23:37:16.048310+0000","last_became_active":"2024-09-15T23:37:16.039622+0000","last_became_peered":"2024-09-15T23:37:16.039622+0000","last_unstale":"2024-09-15T23:37:16.048310+0000","last_undegraded":"2024-09-15T23:37:16.048310+0000","last_fullsized":"2024-09-15T23:37:16.048310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:54.978065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035781+0000","last_change":"2024-09-15T23:37:16.032318+0000","last_active":"2024-09-15T23:37:16.035781+0000","last_peered":"2024-09-15T23:37:16.035781+0000","last_clean":"2024-09-15T23:37:16.035781+0000","last_became_active":"2024-09-15T23:37:16.028787+0000","last_became_peered":"2024-09-15T23:37:16.028787+0000","last_unstale":"2024-09-15T23:37:16.035781+0000","last_undegraded":"2024-09-15T23:37:16.035781+0000","last_fullsized":"2024-09-15T23:37:16.035781+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:31.065835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035685+0000","last_change":"2024-09-15T23:37:16.028365+0000","last_active":"2024-09-15T23:37:16.035685+0000","last_peered":"2024-09-15T23:37:16.035685+0000","last_clean":"2024-09-15T23:37:16.035685+0000","last_became_active":"2024-09-15T23:37:16.028051+0000","last_became_peered":"2024-09-15T23:37:16.028051+0000","last_unstale":"2024-09-15T23:37:16.035685+0000","last_undegraded":"2024-09-15T23:37:16.035685+0000","last_fullsized":"2024-09-15T23:37:16.035685+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:47.439397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.048225+0000","last_change":"2024-09-15T23:37:16.039870+0000","last_active":"2024-09-15T23:37:16.048225+0000","last_peered":"2024-09-15T23:37:16.048225+0000","last_clean":"2024-09-15T23:37:16.048225+0000","last_became_active":"2024-09-15T23:37:16.039295+0000","last_became_peered":"2024-09-15T23:37:16.039295+0000","last_unstale":"2024-09-15T23:37:16.048225+0000","last_undegraded":"2024-09-15T23:37:16.048225+0000","last_fullsized":"2024-09-15T23:37:16.048225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:16.959191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031203+0000","last_change":"2024-09-15T23:37:16.049435+0000","last_active":"2024-09-15T23:37:17.031203+0000","last_peered":"2024-09-15T23:37:17.031203+0000","last_clean":"2024-09-15T23:37:17.031203+0000","last_became_active":"2024-09-15T23:37:16.049084+0000","last_became_peered":"2024-09-15T23:37:16.049084+0000","last_unstale":"2024-09-15T23:37:17.031203+0000","last_undegraded":"2024-09-15T23:37:17.031203+0000","last_fullsized":"2024-09-15T23:37:17.031203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:15:01.569356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046178+0000","last_change":"2024-09-15T23:37:16.044114+0000","last_active":"2024-09-15T23:37:16.046178+0000","last_peered":"2024-09-15T23:37:16.046178+0000","last_clean":"2024-09-15T23:37:16.046178+0000","last_became_active":"2024-09-15T23:37:16.042005+0000","last_became_peered":"2024-09-15T23:37:16.042005+0000","last_unstale":"2024-09-15T23:37:16.046178+0000","last_undegraded":"2024-09-15T23:37:16.046178+0000","last_fullsized":"2024-09-15T23:37:16.046178+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:40.456325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056140+0000","last_change":"2024-09-15T23:37:15.019375+0000","last_active":"2024-09-15T23:37:16.056140+0000","last_peered":"2024-09-15T23:37:16.056140+0000","last_clean":"2024-09-15T23:37:16.056140+0000","last_became_active":"2024-09-15T23:37:15.018977+0000","last_became_peered":"2024-09-15T23:37:15.018977+0000","last_unstale":"2024-09-15T23:37:16.056140+0000","last_undegraded":"2024-09-15T23:37:16.056140+0000","last_fullsized":"2024-09-15T23:37:16.056140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:02.654673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030759+0000","last_change":"2024-09-15T23:37:16.035321+0000","last_active":"2024-09-15T23:37:17.030759+0000","last_peered":"2024-09-15T23:37:17.030759+0000","last_clean":"2024-09-15T23:37:17.030759+0000","last_became_active":"2024-09-15T23:37:16.034225+0000","last_became_peered":"2024-09-15T23:37:16.034225+0000","last_unstale":"2024-09-15T23:37:17.030759+0000","last_undegraded":"2024-09-15T23:37:17.030759+0000","last_fullsized":"2024-09-15T23:37:17.030759+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:23.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031159+0000","last_change":"2024-09-15T23:37:15.024812+0000","last_active":"2024-09-15T23:37:17.031159+0000","last_peered":"2024-09-15T23:37:17.031159+0000","last_clean":"2024-09-15T23:37:17.031159+0000","last_became_active":"2024-09-15T23:37:15.024533+0000","last_became_peered":"2024-09-15T23:37:15.024533+0000","last_unstale":"2024-09-15T23:37:17.031159+0000","last_undegraded":"2024-09-15T23:37:17.031159+0000","last_fullsized":"2024-09-15T23:37:17.031159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:48.084586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.046817+0000","last_change":"2024-09-15T23:37:16.043209+0000","last_active":"2024-09-15T23:37:16.046817+0000","last_peered":"2024-09-15T23:37:16.046817+0000","last_clean":"2024-09-15T23:37:16.046817+0000","last_became_active":"2024-09-15T23:37:16.042016+0000","last_became_peered":"2024-09-15T23:37:16.042016+0000","last_unstale":"2024-09-15T23:37:16.046817+0000","last_undegraded":"2024-09-15T23:37:16.046817+0000","last_fullsized":"2024-09-15T23:37:16.046817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:17.167815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.032084+0000","last_change":"2024-09-15T23:37:15.022564+0000","last_active":"2024-09-15T23:37:17.032084+0000","last_peered":"2024-09-15T23:37:17.032084+0000","last_clean":"2024-09-15T23:37:17.032084+0000","last_became_active":"2024-09-15T23:37:15.021998+0000","last_became_peered":"2024-09-15T23:37:15.021998+0000","last_unstale":"2024-09-15T23:37:17.032084+0000","last_undegraded":"2024-09-15T23:37:17.032084+0000","last_fullsized":"2024-09-15T23:37:17.032084+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:56.759572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031373+0000","last_change":"2024-09-15T23:37:16.035304+0000","last_active":"2024-09-15T23:37:17.031373+0000","last_peered":"2024-09-15T23:37:17.031373+0000","last_clean":"2024-09-15T23:37:17.031373+0000","last_became_active":"2024-09-15T23:37:16.033864+0000","last_became_peered":"2024-09-15T23:37:16.033864+0000","last_unstale":"2024-09-15T23:37:17.031373+0000","last_undegraded":"2024-09-15T23:37:17.031373+0000","last_fullsized":"2024-09-15T23:37:17.031373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:42.504949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.056405+0000","last_change":"2024-09-15T23:37:15.023109+0000","last_active":"2024-09-15T23:37:16.056405+0000","last_peered":"2024-09-15T23:37:16.056405+0000","last_clean":"2024-09-15T23:37:16.056405+0000","last_became_active":"2024-09-15T23:37:15.022494+0000","last_became_peered":"2024-09-15T23:37:15.022494+0000","last_unstale":"2024-09-15T23:37:16.056405+0000","last_undegraded":"2024-09-15T23:37:16.056405+0000","last_fullsized":"2024-09-15T23:37:16.056405+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:29.332641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031227+0000","last_change":"2024-09-15T23:37:16.039406+0000","last_active":"2024-09-15T23:37:17.031227+0000","last_peered":"2024-09-15T23:37:17.031227+0000","last_clean":"2024-09-15T23:37:17.031227+0000","last_became_active":"2024-09-15T23:37:16.039113+0000","last_became_peered":"2024-09-15T23:37:16.039113+0000","last_unstale":"2024-09-15T23:37:17.031227+0000","last_undegraded":"2024-09-15T23:37:17.031227+0000","last_fullsized":"2024-09-15T23:37:17.031227+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:18.555803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.055917+0000","last_change":"2024-09-15T23:37:15.025209+0000","last_active":"2024-09-15T23:37:16.055917+0000","last_peered":"2024-09-15T23:37:16.055917+0000","last_clean":"2024-09-15T23:37:16.055917+0000","last_became_active":"2024-09-15T23:37:15.022678+0000","last_became_peered":"2024-09-15T23:37:15.022678+0000","last_unstale":"2024-09-15T23:37:16.055917+0000","last_undegraded":"2024-09-15T23:37:16.055917+0000","last_fullsized":"2024-09-15T23:37:16.055917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:01.000785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033653+0000","last_change":"2024-09-15T23:37:16.031990+0000","last_active":"2024-09-15T23:37:16.033653+0000","last_peered":"2024-09-15T23:37:16.033653+0000","last_clean":"2024-09-15T23:37:16.033653+0000","last_became_active":"2024-09-15T23:37:16.031688+0000","last_became_peered":"2024-09-15T23:37:16.031688+0000","last_unstale":"2024-09-15T23:37:16.033653+0000","last_undegraded":"2024-09-15T23:37:16.033653+0000","last_fullsized":"2024-09-15T23:37:16.033653+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:36.654534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052263+0000","last_change":"2024-09-15T23:37:15.029001+0000","last_active":"2024-09-15T23:37:16.052263+0000","last_peered":"2024-09-15T23:37:16.052263+0000","last_clean":"2024-09-15T23:37:16.052263+0000","last_became_active":"2024-09-15T23:37:15.028669+0000","last_became_peered":"2024-09-15T23:37:15.028669+0000","last_unstale":"2024-09-15T23:37:16.052263+0000","last_undegraded":"2024-09-15T23:37:16.052263+0000","last_fullsized":"2024-09-15T23:37:16.052263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:32.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035709+0000","last_change":"2024-09-15T23:37:16.034125+0000","last_active":"2024-09-15T23:37:16.035709+0000","last_peered":"2024-09-15T23:37:16.035709+0000","last_clean":"2024-09-15T23:37:16.035709+0000","last_became_active":"2024-09-15T23:37:16.033801+0000","last_became_peered":"2024-09-15T23:37:16.033801+0000","last_unstale":"2024-09-15T23:37:16.035709+0000","last_undegraded":"2024-09-15T23:37:16.035709+0000","last_fullsized":"2024-09-15T23:37:16.035709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:04.589184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031308+0000","last_change":"2024-09-15T23:37:15.027671+0000","last_active":"2024-09-15T23:37:17.031308+0000","last_peered":"2024-09-15T23:37:17.031308+0000","last_clean":"2024-09-15T23:37:17.031308+0000","last_became_active":"2024-09-15T23:37:15.027394+0000","last_became_peered":"2024-09-15T23:37:15.027394+0000","last_unstale":"2024-09-15T23:37:17.031308+0000","last_undegraded":"2024-09-15T23:37:17.031308+0000","last_fullsized":"2024-09-15T23:37:17.031308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:25.898422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030846+0000","last_change":"2024-09-15T23:37:16.041972+0000","last_active":"2024-09-15T23:37:17.030846+0000","last_peered":"2024-09-15T23:37:17.030846+0000","last_clean":"2024-09-15T23:37:17.030846+0000","last_became_active":"2024-09-15T23:37:16.041636+0000","last_became_peered":"2024-09-15T23:37:16.041636+0000","last_unstale":"2024-09-15T23:37:17.030846+0000","last_undegraded":"2024-09-15T23:37:17.030846+0000","last_fullsized":"2024-09-15T23:37:17.030846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:56.834193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022594+0000","last_change":"2024-09-15T23:37:15.026937+0000","last_active":"2024-09-15T23:37:16.022594+0000","last_peered":"2024-09-15T23:37:16.022594+0000","last_clean":"2024-09-15T23:37:16.022594+0000","last_became_active":"2024-09-15T23:37:15.026293+0000","last_became_peered":"2024-09-15T23:37:15.026293+0000","last_unstale":"2024-09-15T23:37:16.022594+0000","last_undegraded":"2024-09-15T23:37:16.022594+0000","last_fullsized":"2024-09-15T23:37:16.022594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:29.524303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031977+0000","last_change":"2024-09-15T23:37:16.037328+0000","last_active":"2024-09-15T23:37:17.031977+0000","last_peered":"2024-09-15T23:37:17.031977+0000","last_clean":"2024-09-15T23:37:17.031977+0000","last_became_active":"2024-09-15T23:37:16.037062+0000","last_became_peered":"2024-09-15T23:37:16.037062+0000","last_unstale":"2024-09-15T23:37:17.031977+0000","last_undegraded":"2024-09-15T23:37:17.031977+0000","last_fullsized":"2024-09-15T23:37:17.031977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:34.486024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.032017+0000","last_change":"2024-09-15T23:37:15.023332+0000","last_active":"2024-09-15T23:37:17.032017+0000","last_peered":"2024-09-15T23:37:17.032017+0000","last_clean":"2024-09-15T23:37:17.032017+0000","last_became_active":"2024-09-15T23:37:15.022455+0000","last_became_peered":"2024-09-15T23:37:15.022455+0000","last_unstale":"2024-09-15T23:37:17.032017+0000","last_undegraded":"2024-09-15T23:37:17.032017+0000","last_fullsized":"2024-09-15T23:37:17.032017+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:07.745774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.045715+0000","last_change":"2024-09-15T23:37:16.044875+0000","last_active":"2024-09-15T23:37:16.045715+0000","last_peered":"2024-09-15T23:37:16.045715+0000","last_clean":"2024-09-15T23:37:16.045715+0000","last_became_active":"2024-09-15T23:37:16.044600+0000","last_became_peered":"2024-09-15T23:37:16.044600+0000","last_unstale":"2024-09-15T23:37:16.045715+0000","last_undegraded":"2024-09-15T23:37:16.045715+0000","last_fullsized":"2024-09-15T23:37:16.045715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:32.153327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020234+0000","last_change":"2024-09-15T23:37:15.025268+0000","last_active":"2024-09-15T23:37:16.020234+0000","last_peered":"2024-09-15T23:37:16.020234+0000","last_clean":"2024-09-15T23:37:16.020234+0000","last_became_active":"2024-09-15T23:37:15.024965+0000","last_became_peered":"2024-09-15T23:37:15.024965+0000","last_unstale":"2024-09-15T23:37:16.020234+0000","last_undegraded":"2024-09-15T23:37:16.020234+0000","last_fullsized":"2024-09-15T23:37:16.020234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:41.312641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.032200+0000","last_change":"2024-09-15T23:37:16.035432+0000","last_active":"2024-09-15T23:37:17.032200+0000","last_peered":"2024-09-15T23:37:17.032200+0000","last_clean":"2024-09-15T23:37:17.032200+0000","last_became_active":"2024-09-15T23:37:16.034615+0000","last_became_peered":"2024-09-15T23:37:16.034615+0000","last_unstale":"2024-09-15T23:37:17.032200+0000","last_undegraded":"2024-09-15T23:37:17.032200+0000","last_fullsized":"2024-09-15T23:37:17.032200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:41.203020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.053070+0000","last_change":"2024-09-15T23:37:15.025534+0000","last_active":"2024-09-15T23:37:16.053070+0000","last_peered":"2024-09-15T23:37:16.053070+0000","last_clean":"2024-09-15T23:37:16.053070+0000","last_became_active":"2024-09-15T23:37:15.023762+0000","last_became_peered":"2024-09-15T23:37:15.023762+0000","last_unstale":"2024-09-15T23:37:16.053070+0000","last_undegraded":"2024-09-15T23:37:16.053070+0000","last_fullsized":"2024-09-15T23:37:16.053070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:11.328653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051853+0000","last_change":"2024-09-15T23:37:16.043056+0000","last_active":"2024-09-15T23:37:16.051853+0000","last_peered":"2024-09-15T23:37:16.051853+0000","last_clean":"2024-09-15T23:37:16.051853+0000","last_became_active":"2024-09-15T23:37:16.042639+0000","last_became_peered":"2024-09-15T23:37:16.042639+0000","last_unstale":"2024-09-15T23:37:16.051853+0000","last_undegraded":"2024-09-15T23:37:16.051853+0000","last_fullsized":"2024-09-15T23:37:16.051853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:51.445196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031852+0000","last_change":"2024-09-15T23:37:15.027048+0000","last_active":"2024-09-15T23:37:17.031852+0000","last_peered":"2024-09-15T23:37:17.031852+0000","last_clean":"2024-09-15T23:37:17.031852+0000","last_became_active":"2024-09-15T23:37:15.026617+0000","last_became_peered":"2024-09-15T23:37:15.026617+0000","last_unstale":"2024-09-15T23:37:17.031852+0000","last_undegraded":"2024-09-15T23:37:17.031852+0000","last_fullsized":"2024-09-15T23:37:17.031852+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:29.053196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031362+0000","last_change":"2024-09-15T23:37:16.042516+0000","last_active":"2024-09-15T23:37:17.031362+0000","last_peered":"2024-09-15T23:37:17.031362+0000","last_clean":"2024-09-15T23:37:17.031362+0000","last_became_active":"2024-09-15T23:37:16.042208+0000","last_became_peered":"2024-09-15T23:37:16.042208+0000","last_unstale":"2024-09-15T23:37:17.031362+0000","last_undegraded":"2024-09-15T23:37:17.031362+0000","last_fullsized":"2024-09-15T23:37:17.031362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:28.104059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030500+0000","last_change":"2024-09-15T23:37:15.022302+0000","last_active":"2024-09-15T23:37:17.030500+0000","last_peered":"2024-09-15T23:37:17.030500+0000","last_clean":"2024-09-15T23:37:17.030500+0000","last_became_active":"2024-09-15T23:37:15.021774+0000","last_became_peered":"2024-09-15T23:37:15.021774+0000","last_unstale":"2024-09-15T23:37:17.030500+0000","last_undegraded":"2024-09-15T23:37:17.030500+0000","last_fullsized":"2024-09-15T23:37:17.030500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:06.302256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030285+0000","last_change":"2024-09-15T23:37:16.036488+0000","last_active":"2024-09-15T23:37:17.030285+0000","last_peered":"2024-09-15T23:37:17.030285+0000","last_clean":"2024-09-15T23:37:17.030285+0000","last_became_active":"2024-09-15T23:37:16.035135+0000","last_became_peered":"2024-09-15T23:37:16.035135+0000","last_unstale":"2024-09-15T23:37:17.030285+0000","last_undegraded":"2024-09-15T23:37:17.030285+0000","last_fullsized":"2024-09-15T23:37:17.030285+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:30.804394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030279+0000","last_change":"2024-09-15T23:37:15.021306+0000","last_active":"2024-09-15T23:37:17.030279+0000","last_peered":"2024-09-15T23:37:17.030279+0000","last_clean":"2024-09-15T23:37:17.030279+0000","last_became_active":"2024-09-15T23:37:15.020973+0000","last_became_peered":"2024-09-15T23:37:15.020973+0000","last_unstale":"2024-09-15T23:37:17.030279+0000","last_undegraded":"2024-09-15T23:37:17.030279+0000","last_fullsized":"2024-09-15T23:37:17.030279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:01.000785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.034232+0000","last_change":"2024-09-15T23:37:16.028516+0000","last_active":"2024-09-15T23:37:16.034232+0000","last_peered":"2024-09-15T23:37:16.034232+0000","last_clean":"2024-09-15T23:37:16.034232+0000","last_became_active":"2024-09-15T23:37:16.028203+0000","last_became_peered":"2024-09-15T23:37:16.028203+0000","last_unstale":"2024-09-15T23:37:16.034232+0000","last_undegraded":"2024-09-15T23:37:16.034232+0000","last_fullsized":"2024-09-15T23:37:16.034232+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:39.454484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020739+0000","last_change":"2024-09-15T23:37:15.020887+0000","last_active":"2024-09-15T23:37:16.020739+0000","last_peered":"2024-09-15T23:37:16.020739+0000","last_clean":"2024-09-15T23:37:16.020739+0000","last_became_active":"2024-09-15T23:37:15.020611+0000","last_became_peered":"2024-09-15T23:37:15.020611+0000","last_unstale":"2024-09-15T23:37:16.020739+0000","last_undegraded":"2024-09-15T23:37:16.020739+0000","last_fullsized":"2024-09-15T23:37:16.020739+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:55.272773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.034496+0000","last_change":"2024-09-15T23:37:16.030057+0000","last_active":"2024-09-15T23:37:17.034496+0000","last_peered":"2024-09-15T23:37:17.034496+0000","last_clean":"2024-09-15T23:37:17.034496+0000","last_became_active":"2024-09-15T23:37:16.029759+0000","last_became_peered":"2024-09-15T23:37:16.029759+0000","last_unstale":"2024-09-15T23:37:17.034496+0000","last_undegraded":"2024-09-15T23:37:17.034496+0000","last_fullsized":"2024-09-15T23:37:17.034496+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:02.005721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.058124+0000","last_change":"2024-09-15T23:37:15.018587+0000","last_active":"2024-09-15T23:37:16.058124+0000","last_peered":"2024-09-15T23:37:16.058124+0000","last_clean":"2024-09-15T23:37:16.058124+0000","last_became_active":"2024-09-15T23:37:15.018296+0000","last_became_peered":"2024-09-15T23:37:15.018296+0000","last_unstale":"2024-09-15T23:37:16.058124+0000","last_undegraded":"2024-09-15T23:37:16.058124+0000","last_fullsized":"2024-09-15T23:37:16.058124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:01.784529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031112+0000","last_change":"2024-09-15T23:37:16.049263+0000","last_active":"2024-09-15T23:37:17.031112+0000","last_peered":"2024-09-15T23:37:17.031112+0000","last_clean":"2024-09-15T23:37:17.031112+0000","last_became_active":"2024-09-15T23:37:16.048955+0000","last_became_peered":"2024-09-15T23:37:16.048955+0000","last_unstale":"2024-09-15T23:37:17.031112+0000","last_undegraded":"2024-09-15T23:37:17.031112+0000","last_fullsized":"2024-09-15T23:37:17.031112+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:01.512554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052687+0000","last_change":"2024-09-15T23:37:15.025903+0000","last_active":"2024-09-15T23:37:16.052687+0000","last_peered":"2024-09-15T23:37:16.052687+0000","last_clean":"2024-09-15T23:37:16.052687+0000","last_became_active":"2024-09-15T23:37:15.025212+0000","last_became_peered":"2024-09-15T23:37:15.025212+0000","last_unstale":"2024-09-15T23:37:16.052687+0000","last_undegraded":"2024-09-15T23:37:16.052687+0000","last_fullsized":"2024-09-15T23:37:16.052687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:41.537186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047929+0000","last_change":"2024-09-15T23:37:16.042373+0000","last_active":"2024-09-15T23:37:16.047929+0000","last_peered":"2024-09-15T23:37:16.047929+0000","last_clean":"2024-09-15T23:37:16.047929+0000","last_became_active":"2024-09-15T23:37:16.040111+0000","last_became_peered":"2024-09-15T23:37:16.040111+0000","last_unstale":"2024-09-15T23:37:16.047929+0000","last_undegraded":"2024-09-15T23:37:16.047929+0000","last_fullsized":"2024-09-15T23:37:16.047929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:36:14.588537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052462+0000","last_change":"2024-09-15T23:37:15.024606+0000","last_active":"2024-09-15T23:37:16.052462+0000","last_peered":"2024-09-15T23:37:16.052462+0000","last_clean":"2024-09-15T23:37:16.052462+0000","last_became_active":"2024-09-15T23:37:15.024349+0000","last_became_peered":"2024-09-15T23:37:15.024349+0000","last_unstale":"2024-09-15T23:37:16.052462+0000","last_undegraded":"2024-09-15T23:37:16.052462+0000","last_fullsized":"2024-09-15T23:37:16.052462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:23.692350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.047894+0000","last_change":"2024-09-15T23:37:16.041862+0000","last_active":"2024-09-15T23:37:16.047894+0000","last_peered":"2024-09-15T23:37:16.047894+0000","last_clean":"2024-09-15T23:37:16.047894+0000","last_became_active":"2024-09-15T23:37:16.041522+0000","last_became_peered":"2024-09-15T23:37:16.041522+0000","last_unstale":"2024-09-15T23:37:16.047894+0000","last_undegraded":"2024-09-15T23:37:16.047894+0000","last_fullsized":"2024-09-15T23:37:16.047894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:08.290613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.034471+0000","last_change":"2024-09-15T23:37:15.023432+0000","last_active":"2024-09-15T23:37:17.034471+0000","last_peered":"2024-09-15T23:37:17.034471+0000","last_clean":"2024-09-15T23:37:17.034471+0000","last_became_active":"2024-09-15T23:37:15.023142+0000","last_became_peered":"2024-09-15T23:37:15.023142+0000","last_unstale":"2024-09-15T23:37:17.034471+0000","last_undegraded":"2024-09-15T23:37:17.034471+0000","last_fullsized":"2024-09-15T23:37:17.034471+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:28.655175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.897161+0000","last_change":"2024-09-15T23:37:16.040051+0000","last_active":"2024-09-15T23:37:17.897161+0000","last_peered":"2024-09-15T23:37:17.897161+0000","last_clean":"2024-09-15T23:37:17.897161+0000","last_became_active":"2024-09-15T23:37:16.039757+0000","last_became_peered":"2024-09-15T23:37:16.039757+0000","last_unstale":"2024-09-15T23:37:17.897161+0000","last_undegraded":"2024-09-15T23:37:17.897161+0000","last_fullsized":"2024-09-15T23:37:17.897161+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:57.034921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.896841+0000","last_change":"2024-09-15T23:37:15.027655+0000","last_active":"2024-09-15T23:37:17.896841+0000","last_peered":"2024-09-15T23:37:17.896841+0000","last_clean":"2024-09-15T23:37:17.896841+0000","last_became_active":"2024-09-15T23:37:15.027352+0000","last_became_peered":"2024-09-15T23:37:15.027352+0000","last_unstale":"2024-09-15T23:37:17.896841+0000","last_undegraded":"2024-09-15T23:37:17.896841+0000","last_fullsized":"2024-09-15T23:37:17.896841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:29.053196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.025069+0000","last_change":"2024-09-15T23:37:16.023795+0000","last_active":"2024-09-15T23:37:16.025069+0000","last_peered":"2024-09-15T23:37:16.025069+0000","last_clean":"2024-09-15T23:37:16.025069+0000","last_became_active":"2024-09-15T23:37:16.023493+0000","last_became_peered":"2024-09-15T23:37:16.023493+0000","last_unstale":"2024-09-15T23:37:16.025069+0000","last_undegraded":"2024-09-15T23:37:16.025069+0000","last_fullsized":"2024-09-15T23:37:16.025069+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:19.252190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.896865+0000","last_change":"2024-09-15T23:37:15.023359+0000","last_active":"2024-09-15T23:37:17.896865+0000","last_peered":"2024-09-15T23:37:17.896865+0000","last_clean":"2024-09-15T23:37:17.896865+0000","last_became_active":"2024-09-15T23:37:15.023051+0000","last_became_peered":"2024-09-15T23:37:15.023051+0000","last_unstale":"2024-09-15T23:37:17.896865+0000","last_undegraded":"2024-09-15T23:37:17.896865+0000","last_fullsized":"2024-09-15T23:37:17.896865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.834071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030921+0000","last_change":"2024-09-15T23:37:16.050409+0000","last_active":"2024-09-15T23:37:17.030921+0000","last_peered":"2024-09-15T23:37:17.030921+0000","last_clean":"2024-09-15T23:37:17.030921+0000","last_became_active":"2024-09-15T23:37:16.049904+0000","last_became_peered":"2024-09-15T23:37:16.049904+0000","last_unstale":"2024-09-15T23:37:17.030921+0000","last_undegraded":"2024-09-15T23:37:17.030921+0000","last_fullsized":"2024-09-15T23:37:17.030921+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:56.174557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030177+0000","last_change":"2024-09-15T23:37:15.020340+0000","last_active":"2024-09-15T23:37:17.030177+0000","last_peered":"2024-09-15T23:37:17.030177+0000","last_clean":"2024-09-15T23:37:17.030177+0000","last_became_active":"2024-09-15T23:37:15.020031+0000","last_became_peered":"2024-09-15T23:37:15.020031+0000","last_unstale":"2024-09-15T23:37:17.030177+0000","last_undegraded":"2024-09-15T23:37:17.030177+0000","last_fullsized":"2024-09-15T23:37:17.030177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:29.332641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.051420+0000","last_change":"2024-09-15T23:37:16.049420+0000","last_active":"2024-09-15T23:37:16.051420+0000","last_peered":"2024-09-15T23:37:16.051420+0000","last_clean":"2024-09-15T23:37:16.051420+0000","last_became_active":"2024-09-15T23:37:16.049125+0000","last_became_peered":"2024-09-15T23:37:16.049125+0000","last_unstale":"2024-09-15T23:37:16.051420+0000","last_undegraded":"2024-09-15T23:37:16.051420+0000","last_fullsized":"2024-09-15T23:37:16.051420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:56.834193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.024060+0000","last_change":"2024-09-15T23:37:15.030423+0000","last_active":"2024-09-15T23:37:16.024060+0000","last_peered":"2024-09-15T23:37:16.024060+0000","last_clean":"2024-09-15T23:37:16.024060+0000","last_became_active":"2024-09-15T23:37:15.030077+0000","last_became_peered":"2024-09-15T23:37:15.030077+0000","last_unstale":"2024-09-15T23:37:16.024060+0000","last_undegraded":"2024-09-15T23:37:16.024060+0000","last_fullsized":"2024-09-15T23:37:16.024060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.262938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030469+0000","last_change":"2024-09-15T23:37:16.048040+0000","last_active":"2024-09-15T23:37:17.030469+0000","last_peered":"2024-09-15T23:37:17.030469+0000","last_clean":"2024-09-15T23:37:17.030469+0000","last_became_active":"2024-09-15T23:37:16.047733+0000","last_became_peered":"2024-09-15T23:37:16.047733+0000","last_unstale":"2024-09-15T23:37:17.030469+0000","last_undegraded":"2024-09-15T23:37:17.030469+0000","last_fullsized":"2024-09-15T23:37:17.030469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:19.288004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.029958+0000","last_change":"2024-09-15T23:37:15.018265+0000","last_active":"2024-09-15T23:37:17.029958+0000","last_peered":"2024-09-15T23:37:17.029958+0000","last_clean":"2024-09-15T23:37:17.029958+0000","last_became_active":"2024-09-15T23:37:15.017961+0000","last_became_peered":"2024-09-15T23:37:15.017961+0000","last_unstale":"2024-09-15T23:37:17.029958+0000","last_undegraded":"2024-09-15T23:37:17.029958+0000","last_fullsized":"2024-09-15T23:37:17.029958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:25.820835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.034541+0000","last_change":"2024-09-15T23:37:16.044084+0000","last_active":"2024-09-15T23:37:17.034541+0000","last_peered":"2024-09-15T23:37:17.034541+0000","last_clean":"2024-09-15T23:37:17.034541+0000","last_became_active":"2024-09-15T23:37:16.043790+0000","last_became_peered":"2024-09-15T23:37:16.043790+0000","last_unstale":"2024-09-15T23:37:17.034541+0000","last_undegraded":"2024-09-15T23:37:17.034541+0000","last_fullsized":"2024-09-15T23:37:17.034541+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:30.337576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052977+0000","last_change":"2024-09-15T23:37:15.021730+0000","last_active":"2024-09-15T23:37:16.052977+0000","last_peered":"2024-09-15T23:37:16.052977+0000","last_clean":"2024-09-15T23:37:16.052977+0000","last_became_active":"2024-09-15T23:37:15.021217+0000","last_became_peered":"2024-09-15T23:37:15.021217+0000","last_unstale":"2024-09-15T23:37:16.052977+0000","last_undegraded":"2024-09-15T23:37:16.052977+0000","last_fullsized":"2024-09-15T23:37:16.052977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:43:54.440674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.052881+0000","last_change":"2024-09-15T23:37:16.050557+0000","last_active":"2024-09-15T23:37:16.052881+0000","last_peered":"2024-09-15T23:37:16.052881+0000","last_clean":"2024-09-15T23:37:16.052881+0000","last_became_active":"2024-09-15T23:37:16.050267+0000","last_became_peered":"2024-09-15T23:37:16.050267+0000","last_unstale":"2024-09-15T23:37:16.052881+0000","last_undegraded":"2024-09-15T23:37:16.052881+0000","last_fullsized":"2024-09-15T23:37:16.052881+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:01.512554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.034628+0000","last_change":"2024-09-15T23:37:15.029187+0000","last_active":"2024-09-15T23:37:17.034628+0000","last_peered":"2024-09-15T23:37:17.034628+0000","last_clean":"2024-09-15T23:37:17.034628+0000","last_became_active":"2024-09-15T23:37:15.028731+0000","last_became_peered":"2024-09-15T23:37:15.028731+0000","last_unstale":"2024-09-15T23:37:17.034628+0000","last_undegraded":"2024-09-15T23:37:17.034628+0000","last_fullsized":"2024-09-15T23:37:17.034628+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:15.954255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.897042+0000","last_change":"2024-09-15T23:37:16.036655+0000","last_active":"2024-09-15T23:37:17.897042+0000","last_peered":"2024-09-15T23:37:17.897042+0000","last_clean":"2024-09-15T23:37:17.897042+0000","last_became_active":"2024-09-15T23:37:16.036323+0000","last_became_peered":"2024-09-15T23:37:16.036323+0000","last_unstale":"2024-09-15T23:37:17.897042+0000","last_undegraded":"2024-09-15T23:37:17.897042+0000","last_fullsized":"2024-09-15T23:37:17.897042+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:25.504668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.034531+0000","last_change":"2024-09-15T23:37:15.028765+0000","last_active":"2024-09-15T23:37:17.034531+0000","last_peered":"2024-09-15T23:37:17.034531+0000","last_clean":"2024-09-15T23:37:17.034531+0000","last_became_active":"2024-09-15T23:37:15.028426+0000","last_became_peered":"2024-09-15T23:37:15.028426+0000","last_unstale":"2024-09-15T23:37:17.034531+0000","last_undegraded":"2024-09-15T23:37:17.034531+0000","last_fullsized":"2024-09-15T23:37:17.034531+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:50.540129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.034286+0000","last_change":"2024-09-15T23:37:16.041075+0000","last_active":"2024-09-15T23:37:17.034286+0000","last_peered":"2024-09-15T23:37:17.034286+0000","last_clean":"2024-09-15T23:37:17.034286+0000","last_became_active":"2024-09-15T23:37:16.040367+0000","last_became_peered":"2024-09-15T23:37:16.040367+0000","last_unstale":"2024-09-15T23:37:17.034286+0000","last_undegraded":"2024-09-15T23:37:17.034286+0000","last_fullsized":"2024-09-15T23:37:17.034286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:26.825771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031543+0000","last_change":"2024-09-15T23:37:15.021219+0000","last_active":"2024-09-15T23:37:17.031543+0000","last_peered":"2024-09-15T23:37:17.031543+0000","last_clean":"2024-09-15T23:37:17.031543+0000","last_became_active":"2024-09-15T23:37:15.020887+0000","last_became_peered":"2024-09-15T23:37:15.020887+0000","last_unstale":"2024-09-15T23:37:17.031543+0000","last_undegraded":"2024-09-15T23:37:17.031543+0000","last_fullsized":"2024-09-15T23:37:17.031543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.834071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.030454+0000","last_change":"2024-09-15T23:37:16.036221+0000","last_active":"2024-09-15T23:37:17.030454+0000","last_peered":"2024-09-15T23:37:17.030454+0000","last_clean":"2024-09-15T23:37:17.030454+0000","last_became_active":"2024-09-15T23:37:16.035764+0000","last_became_peered":"2024-09-15T23:37:16.035764+0000","last_unstale":"2024-09-15T23:37:17.030454+0000","last_undegraded":"2024-09-15T23:37:17.030454+0000","last_fullsized":"2024-09-15T23:37:17.030454+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:45.986835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.026104+0000","last_change":"2024-09-15T23:37:15.023867+0000","last_active":"2024-09-15T23:37:16.026104+0000","last_peered":"2024-09-15T23:37:16.026104+0000","last_clean":"2024-09-15T23:37:16.026104+0000","last_became_active":"2024-09-15T23:37:15.023470+0000","last_became_peered":"2024-09-15T23:37:15.023470+0000","last_unstale":"2024-09-15T23:37:16.026104+0000","last_undegraded":"2024-09-15T23:37:16.026104+0000","last_fullsized":"2024-09-15T23:37:16.026104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:13.335910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.044234+0000","last_change":"2024-09-15T23:37:16.041472+0000","last_active":"2024-09-15T23:37:16.044234+0000","last_peered":"2024-09-15T23:37:16.044234+0000","last_clean":"2024-09-15T23:37:16.044234+0000","last_became_active":"2024-09-15T23:37:16.040586+0000","last_became_peered":"2024-09-15T23:37:16.040586+0000","last_unstale":"2024-09-15T23:37:16.044234+0000","last_undegraded":"2024-09-15T23:37:16.044234+0000","last_fullsized":"2024-09-15T23:37:16.044234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:01.946016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.020562+0000","last_change":"2024-09-15T23:37:15.022433+0000","last_active":"2024-09-15T23:37:16.020562+0000","last_peered":"2024-09-15T23:37:16.020562+0000","last_clean":"2024-09-15T23:37:16.020562+0000","last_became_active":"2024-09-15T23:37:15.022150+0000","last_became_peered":"2024-09-15T23:37:15.022150+0000","last_unstale":"2024-09-15T23:37:16.020562+0000","last_undegraded":"2024-09-15T23:37:16.020562+0000","last_fullsized":"2024-09-15T23:37:16.020562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:06.425003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.033731+0000","last_change":"2024-09-15T23:37:16.031076+0000","last_active":"2024-09-15T23:37:16.033731+0000","last_peered":"2024-09-15T23:37:16.033731+0000","last_clean":"2024-09-15T23:37:16.033731+0000","last_became_active":"2024-09-15T23:37:16.030718+0000","last_became_peered":"2024-09-15T23:37:16.030718+0000","last_unstale":"2024-09-15T23:37:16.033731+0000","last_undegraded":"2024-09-15T23:37:16.033731+0000","last_fullsized":"2024-09-15T23:37:16.033731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:31.411462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T23:37:17.031286+0000","last_change":"2024-09-15T23:37:15.026739+0000","last_active":"2024-09-15T23:37:17.031286+0000","last_peered":"2024-09-15T23:37:17.031286+0000","last_clean":"2024-09-15T23:37:17.031286+0000","last_became_active":"2024-09-15T23:37:15.026460+0000","last_became_peered":"2024-09-15T23:37:15.026460+0000","last_unstale":"2024-09-15T23:37:17.031286+0000","last_undegraded":"2024-09-15T23:37:17.031286+0000","last_fullsized":"2024-09-15T23:37:17.031286+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:20.257767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.035661+0000","last_change":"2024-09-15T23:37:16.034326+0000","last_active":"2024-09-15T23:37:16.035661+0000","last_peered":"2024-09-15T23:37:16.035661+0000","last_clean":"2024-09-15T23:37:16.035661+0000","last_became_active":"2024-09-15T23:37:16.033729+0000","last_became_peered":"2024-09-15T23:37:16.033729+0000","last_unstale":"2024-09-15T23:37:16.035661+0000","last_undegraded":"2024-09-15T23:37:16.035661+0000","last_fullsized":"2024-09-15T23:37:16.035661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:15.003519+0000","last_clean_scrub_stamp":"2024-09-15T23:37:15.003519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:50.331476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.027426+0000","last_change":"2024-09-15T23:37:15.024501+0000","last_active":"2024-09-15T23:37:16.027426+0000","last_peered":"2024-09-15T23:37:16.027426+0000","last_clean":"2024-09-15T23:37:16.027426+0000","last_became_active":"2024-09-15T23:37:15.024195+0000","last_became_peered":"2024-09-15T23:37:15.024195+0000","last_unstale":"2024-09-15T23:37:16.027426+0000","last_undegraded":"2024-09-15T23:37:16.027426+0000","last_fullsized":"2024-09-15T23:37:16.027426+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:37:13.998584+0000","last_clean_scrub_stamp":"2024-09-15T23:37:13.998584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:31.148391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T23:37:16.022665+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:37:16.022665+0000","last_peered":"2024-09-15T23:37:16.022665+0000","last_clean":"2024-09-15T23:37:16.022665+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:16.022665+0000","last_undegraded":"2024-09-15T23:37:16.022665+0000","last_fullsized":"2024-09-15T23:37:16.022665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":185007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":446464,"data_stored":187201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1028096,"data_stored":775397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":438272,"data_stored":185099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":191769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":191733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1040384,"data_stored":782649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":462848,"data_stored":193435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:37:23.751 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T23:37:23.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-15T23:37:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.897+0000 7fb20e2f1640 1 Processor -- start 2024-09-15T23:37:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.897+0000 7fb20e2f1640 1 -- start start 2024-09-15T23:37:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20e2f1640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20815e850 0x7fb2080d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20e2f1640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb208070ea0 0x7fb2080b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20e2f1640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb20809bae0 con 0x7fb20809b450 2024-09-15T23:37:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20e2f1640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb2080d05b0 con 0x7fb208070ea0 2024-09-15T23:37:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20e2f1640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb2080d1110 con 0x7fb20815e850 2024-09-15T23:37:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20c88e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb208070ea0 0x7fb2080b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20c88e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb208070ea0 0x7fb2080b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54586/0 (socket says 172.21.15.7:54586) 2024-09-15T23:37:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20c88e640 1 -- 172.21.15.7:0/3654368996 learned_addr learned my addr 172.21.15.7:0/3654368996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb207fff640 1 --2- 172.21.15.7:0/3654368996 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20815e850 0x7fb2080d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20c88e640 1 -- 172.21.15.7:0/3654368996 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20815e850 msgr2=0x7fb2080d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20c88e640 1 --2- 172.21.15.7:0/3654368996 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20815e850 0x7fb2080d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20c88e640 1 -- 172.21.15.7:0/3654368996 >> v1:172.21.15.18:6789/0 conn(0x7fb20809b450 legacy=0x7fb208070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.898+0000 7fb20c88e640 1 -- 172.21.15.7:0/3654368996 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2080d07e0 con 0x7fb208070ea0 2024-09-15T23:37:23.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.899+0000 7fb20c88e640 1 --2- 172.21.15.7:0/3654368996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208070ea0 0x7fb2080b18c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb200009e50 tx=0x7fb20002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d138fbc2a36ca88 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:23.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.899+0000 7fb206ffd640 1 -- 172.21.15.7:0/3654368996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200036070 con 0x7fb208070ea0 2024-09-15T23:37:23.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.899+0000 7fb206ffd640 1 -- 172.21.15.7:0/3654368996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb200031600 con 0x7fb208070ea0 2024-09-15T23:37:23.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.899+0000 7fb206ffd640 1 -- 172.21.15.7:0/3654368996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000319d0 con 0x7fb208070ea0 2024-09-15T23:37:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.900+0000 7fb20e2f1640 1 -- 172.21.15.7:0/3654368996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208070ea0 msgr2=0x7fb2080b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.900+0000 7fb20e2f1640 1 --2- 172.21.15.7:0/3654368996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208070ea0 0x7fb2080b18c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb200009e50 tx=0x7fb20002a710 comp rx=0 tx=0).stop 2024-09-15T23:37:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.900+0000 7fb20e2f1640 1 -- 172.21.15.7:0/3654368996 shutdown_connections 2024-09-15T23:37:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.900+0000 7fb20e2f1640 1 --2- 172.21.15.7:0/3654368996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208070ea0 0x7fb2080b18c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.900+0000 7fb20e2f1640 1 --2- 172.21.15.7:0/3654368996 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20815e850 0x7fb2080d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.900+0000 7fb20e2f1640 1 -- 172.21.15.7:0/3654368996 >> 172.21.15.7:0/3654368996 conn(0x7fb2080d2420 msgr2=0x7fb2080d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.900+0000 7fb20e2f1640 1 -- 172.21.15.7:0/3654368996 shutdown_connections 2024-09-15T23:37:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.900+0000 7fb20e2f1640 1 -- 172.21.15.7:0/3654368996 wait complete. 2024-09-15T23:37:23.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.901+0000 7fb20e2f1640 1 Processor -- start 2024-09-15T23:37:23.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.901+0000 7fb20e2f1640 1 -- start start 2024-09-15T23:37:23.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb20e2f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208070ea0 0x7fb208157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb20e2f1640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20809b450 0x7fb2080c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb20e2f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb20815e850 0x7fb2081e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb20e2f1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2080d0da0 con 0x7fb20815e850 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb20e2f1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb20809b890 con 0x7fb208070ea0 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb20e2f1640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb20815ec30 con 0x7fb20809b450 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb2077fe640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20809b450 0x7fb2080c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb207fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208070ea0 0x7fb208157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb2077fe640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20809b450 0x7fb2080c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:52548/0 (socket says 172.21.15.7:52548) 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.902+0000 7fb2077fe640 1 -- 172.21.15.7:0/2905293779 learned_addr learned my addr 172.21.15.7:0/2905293779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb20c88e640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb20815e850 0x7fb2081e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb2077fe640 1 -- 172.21.15.7:0/2905293779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208070ea0 msgr2=0x7fb208157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb2077fe640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208070ea0 0x7fb208157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb2077fe640 1 -- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb20815e850 msgr2=0x7fb2081e4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb2077fe640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb20815e850 0x7fb2081e4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb2077fe640 1 -- 172.21.15.7:0/2905293779 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2081e3d40 con 0x7fb20809b450 2024-09-15T23:37:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb207fff640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208070ea0 0x7fb208157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb2077fe640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20809b450 0x7fb2080c8000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00c470 tx=0x7fb1fc001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb204ff9640 1 -- 172.21.15.7:0/2905293779 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0052d0 con 0x7fb20809b450 2024-09-15T23:37:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb204ff9640 1 -- 172.21.15.7:0/2905293779 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1fc005470 con 0x7fb20809b450 2024-09-15T23:37:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb204ff9640 1 -- 172.21.15.7:0/2905293779 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc007260 con 0x7fb20809b450 2024-09-15T23:37:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2081e4020 con 0x7fb20809b450 2024-09-15T23:37:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.903+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081e45e0 con 0x7fb20809b450 2024-09-15T23:37:23.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.904+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2080b18c0 con 0x7fb20809b450 2024-09-15T23:37:23.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.905+0000 7fb204ff9640 1 -- 172.21.15.7:0/2905293779 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb1fc007400 con 0x7fb20809b450 2024-09-15T23:37:23.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.905+0000 7fb204ff9640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1f406c2e0 0x7fb1f406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:23.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.905+0000 7fb204ff9640 1 -- 172.21.15.7:0/2905293779 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fb1fc08ab60 con 0x7fb20809b450 2024-09-15T23:37:23.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.905+0000 7fb207fff640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1f406c2e0 0x7fb1f406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:23.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.906+0000 7fb207fff640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1f406c2e0 0x7fb1f406e7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009e90 tx=0x7fb1f8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:23.910+0000 7fb204ff9640 1 -- 172.21.15.7:0/2905293779 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb1fc059120 con 0x7fb20809b450 2024-09-15T23:37:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.160+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb2080b83a0 con 0x7fb20809b450 2024-09-15T23:37:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.164+0000 7fb204ff9640 1 -- 172.21.15.7:0/2905293779 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7fb1fc058a20 con 0x7fb20809b450 2024-09-15T23:37:24.165 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-15T23:37:24.165 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBAb+dmllwaFhAA/nHwGTNRbiagVIKXM0AeYA== 2024-09-15T23:37:24.165 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-09-15T23:37:24.165 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-09-15T23:37:24.165 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-15T23:37:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-15T23:37:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1f406c2e0 msgr2=0x7fb1f406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1f406c2e0 0x7fb1f406e7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009e90 tx=0x7fb1f8009190 comp rx=0 tx=0).stop 2024-09-15T23:37:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20809b450 msgr2=0x7fb2080c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20809b450 0x7fb2080c8000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00c470 tx=0x7fb1fc001e70 comp rx=0 tx=0).stop 2024-09-15T23:37:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 shutdown_connections 2024-09-15T23:37:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb20815e850 0x7fb2081e4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fb1f406c2e0 0x7fb1f406e7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb20809b450 0x7fb2080c8000 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 --2- 172.21.15.7:0/2905293779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208070ea0 0x7fb208157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 >> 172.21.15.7:0/2905293779 conn(0x7fb2080d2420 msgr2=0x7fb2080704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 shutdown_connections 2024-09-15T23:37:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.166+0000 7fb20e2f1640 1 -- 172.21.15.7:0/2905293779 wait complete. 2024-09-15T23:37:24.178 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:24.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.279+0000 7f6dea1f8640 1 Processor -- start 2024-09-15T23:37:24.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.279+0000 7f6dea1f8640 1 -- start start 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.280+0000 7f6dea1f8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415e6d0 0x7f6de40d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.280+0000 7f6dea1f8640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6de40b3d10 con 0x7f6de40d51c0 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.280+0000 7f6dea1f8640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6de40cfcd0 con 0x7f6de40d32a0 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.280+0000 7f6dea1f8640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6de40cff50 con 0x7f6de415e6d0 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.280+0000 7f6de2ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6de40d51c0 0x7f6de40d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51968/0 (socket says 172.21.15.7:51968) 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.280+0000 7f6de2ffd640 1 -- 172.21.15.7:0/1291007226 learned_addr learned my addr 172.21.15.7:0/1291007226 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.280+0000 7f6de37fe640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415e6d0 0x7f6de40d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.280+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827674124 0 0) 0x7f6de40b3d10 con 0x7f6de40d51c0 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.280+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dcc003520 con 0x7f6de40d51c0 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 453491327 0 0) 0x7f6dcc003520 con 0x7f6de40d51c0 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415e6d0 msgr2=0x7f6de40d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415e6d0 0x7f6de40d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 >> v1:172.21.15.18:6789/0 conn(0x7f6de40d32a0 legacy=0x7f6de40b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de40d0230 con 0x7f6de40d51c0 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f6dd80017f0 con 0x7f6de40d51c0 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 >> v1:172.21.15.7:6789/0 conn(0x7f6de40d51c0 legacy=0x7f6de40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6dcc0039e0 0x7f6dcc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415e6d0 0x7f6dcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6dcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6de40cff50 con 0x7f6dcc004d40 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc00ba70 con 0x7f6dcc0039e0 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6dcc00bc70 con 0x7f6de415e6d0 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de3fff640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6dcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de2ffd640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415e6d0 0x7f6dcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de37fe640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6dcc0039e0 0x7f6dcc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de3fff640 1 -- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415e6d0 msgr2=0x7f6dcc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de3fff640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415e6d0 0x7f6dcc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de3fff640 1 -- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6dcc0039e0 msgr2=0x7f6dcc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de3fff640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6dcc0039e0 0x7f6dcc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.281+0000 7f6de3fff640 1 -- 172.21.15.7:0/1291007226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc003520 con 0x7f6dcc004d40 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.282+0000 7f6de3fff640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6dcc0051c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6de408daa0 tx=0x7f6dcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fa649b25ddb535 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.282+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc001a70 con 0x7f6dcc004d40 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.282+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ddc02add0 con 0x7f6dcc004d40 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.282+0000 7f6de27fc640 1 -- 172.21.15.7:0/1291007226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc003bb0 con 0x7f6dcc004d40 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.283+0000 7f6dea1f8640 1 -- 172.21.15.7:0/1291007226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 msgr2=0x7f6dcc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.283+0000 7f6dea1f8640 1 --2- 172.21.15.7:0/1291007226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6dcc0051c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6de408daa0 tx=0x7f6dcc005830 comp rx=0 tx=0).stop 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.283+0000 7f6de3fff640 1 -- 172.21.15.7:0/1291007226 reap_dead start 2024-09-15T23:37:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.283+0000 7f6dea1f8640 1 -- 172.21.15.7:0/1291007226 shutdown_connections 2024-09-15T23:37:24.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.283+0000 7f6dea1f8640 1 -- 172.21.15.7:0/1291007226 >> 172.21.15.7:0/1291007226 conn(0x7f6de408bfd0 msgr2=0x7f6de40d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:24.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.283+0000 7f6dea1f8640 1 -- 172.21.15.7:0/1291007226 shutdown_connections 2024-09-15T23:37:24.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.283+0000 7f6dea1f8640 1 -- 172.21.15.7:0/1291007226 wait complete. 2024-09-15T23:37:24.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.284+0000 7f6dea1f8640 1 Processor -- start 2024-09-15T23:37:24.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.284+0000 7f6dea1f8640 1 -- start start 2024-09-15T23:37:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6dea1f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6de40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6dea1f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6de41615a0 0x7f6de4161a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6dea1f8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415d190 0x7f6de415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6dea1f8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6de40d21b0 con 0x7f6dcc004d40 2024-09-15T23:37:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6dea1f8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6de40d0510 con 0x7f6de41615a0 2024-09-15T23:37:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6dea1f8640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6de40d2330 con 0x7f6de415d190 2024-09-15T23:37:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6de40b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6de40b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54606/0 (socket says 172.21.15.7:54606) 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de37fe640 1 -- 172.21.15.7:0/3058476596 learned_addr learned my addr 172.21.15.7:0/3058476596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de2ffd640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6de41615a0 0x7f6de4161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de3fff640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415d190 0x7f6de415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de37fe640 1 -- 172.21.15.7:0/3058476596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415d190 msgr2=0x7f6de415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de37fe640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415d190 0x7f6de415d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de37fe640 1 -- 172.21.15.7:0/3058476596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6de41615a0 msgr2=0x7f6de4161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de37fe640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6de41615a0 0x7f6de4161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de37fe640 1 -- 172.21.15.7:0/3058476596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de415db30 con 0x7f6dcc004d40 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de2ffd640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6de41615a0 0x7f6de4161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6de37fe640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6de40b4330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40119a0 tx=0x7f6dd4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.285+0000 7f6dcbfff640 1 -- 172.21.15.7:0/3058476596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400faf0 con 0x7f6dcc004d40 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.286+0000 7f6dcbfff640 1 -- 172.21.15.7:0/3058476596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd400fc90 con 0x7f6dcc004d40 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.286+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de40ca610 con 0x7f6dcc004d40 2024-09-15T23:37:24.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.286+0000 7f6dcbfff640 1 -- 172.21.15.7:0/3058476596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4014440 con 0x7f6dcc004d40 2024-09-15T23:37:24.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.286+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de40cabd0 con 0x7f6dcc004d40 2024-09-15T23:37:24.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.286+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de40d2740 con 0x7f6dcc004d40 2024-09-15T23:37:24.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.287+0000 7f6dcbfff640 1 -- 172.21.15.7:0/3058476596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6dd4012040 con 0x7f6dcc004d40 2024-09-15T23:37:24.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.287+0000 7f6dcbfff640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6dd006c2e0 0x7f6dd006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:24.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.288+0000 7f6dcbfff640 1 -- 172.21.15.7:0/3058476596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6dd408bb40 con 0x7f6dcc004d40 2024-09-15T23:37:24.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.288+0000 7f6de2ffd640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6dd006c2e0 0x7f6dd006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:24.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.288+0000 7f6de2ffd640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6dd006c2e0 0x7f6dd006e7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8000ef0 tx=0x7f6dd8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:24.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.292+0000 7f6dcbfff640 1 -- 172.21.15.7:0/3058476596 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6dd4019030 con 0x7f6dcc004d40 2024-09-15T23:37:24.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.503+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6de40b4d20 con 0x7f6dcc004d40 2024-09-15T23:37:24.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.504+0000 7f6dcbfff640 1 -- 172.21.15.7:0/3058476596 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5214 (secure 0 0 0) 0x7f6dd405a120 con 0x7f6dcc004d40 2024-09-15T23:37:24.505 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:24.505 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"btime":"2024-09-15T23:37:18:028501+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1101755994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6833","nonce":1101755994},{"type":"v1","addr":"172.21.15.18:6835","nonce":1101755994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6834/2816400413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2816400413},{"type":"v1","addr":"172.21.15.18:6834","nonce":2816400413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4827,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/809864756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":809864756},{"type":"v1","addr":"172.21.15.7:6836","nonce":809864756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:37:15.374444+0000","modified":"2024-09-15T23:37:17.368915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4830},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4830":{"gid":4830,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/1729414280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1729414280},{"type":"v1","addr":"172.21.15.7:6837","nonce":1729414280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4830,"qdb_cluster":[4830]},"id":2}]} 2024-09-15T23:37:24.505 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 15 2024-09-15T23:37:24.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6dd006c2e0 msgr2=0x7f6dd006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:24.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6dd006c2e0 0x7f6dd006e7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8000ef0 tx=0x7f6dd8009040 comp rx=0 tx=0).stop 2024-09-15T23:37:24.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 msgr2=0x7f6de40b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:24.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6de40b4330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40119a0 tx=0x7f6dd4011e60 comp rx=0 tx=0).stop 2024-09-15T23:37:24.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 shutdown_connections 2024-09-15T23:37:24.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f6dd006c2e0 0x7f6dd006e7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6de415d190 0x7f6de415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6de41615a0 0x7f6de4161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 --2- 172.21.15.7:0/3058476596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc004d40 0x7f6de40b4330 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:24.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 >> 172.21.15.7:0/3058476596 conn(0x7f6de408bfd0 msgr2=0x7f6de406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:24.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 shutdown_connections 2024-09-15T23:37:24.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:24.506+0000 7f6dea1f8640 1 -- 172.21.15.7:0/3058476596 wait complete. 2024-09-15T23:37:24.922 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-15T23:37:15.374444+0000', 'modified': '2024-09-15T23:37:17.368915+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4830}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4830': {'gid': 4830, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6837/1729414280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6835', 'nonce': 1729414280}, {'type': 'v1', 'addr': '172.21.15.7:6837', 'nonce': 1729414280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4830, 'qdb_cluster': [4830]} 2024-09-15T23:37:24.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-15T23:37:24.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-15T23:37:25.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.029+0000 7fe63e227640 1 Processor -- start 2024-09-15T23:37:25.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.029+0000 7fe63e227640 1 -- start start 2024-09-15T23:37:25.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe63e227640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe638161f20 0x7fe6380d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe63e227640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6380b5c40 0x7fe6380d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe63e227640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6380d4f80 con 0x7fe6380b5570 2024-09-15T23:37:25.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe63e227640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe6380d5540 con 0x7fe6380b5c40 2024-09-15T23:37:25.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe63e227640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe6380d4d30 con 0x7fe638161f20 2024-09-15T23:37:25.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe637fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6380b5c40 0x7fe6380d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe637fff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6380b5c40 0x7fe6380d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54608/0 (socket says 172.21.15.7:54608) 2024-09-15T23:37:25.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe637fff640 1 -- 172.21.15.7:0/2960770867 learned_addr learned my addr 172.21.15.7:0/2960770867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:25.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe6377fe640 1 --2- 172.21.15.7:0/2960770867 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe638161f20 0x7fe6380d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe637fff640 1 -- 172.21.15.7:0/2960770867 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe638161f20 msgr2=0x7fe6380d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe637fff640 1 --2- 172.21.15.7:0/2960770867 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe638161f20 0x7fe6380d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe637fff640 1 -- 172.21.15.7:0/2960770867 >> v1:172.21.15.18:6789/0 conn(0x7fe6380b5570 legacy=0x7fe6380d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.030+0000 7fe637fff640 1 -- 172.21.15.7:0/2960770867 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380d5260 con 0x7fe6380b5c40 2024-09-15T23:37:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.031+0000 7fe637fff640 1 --2- 172.21.15.7:0/2960770867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6380b5c40 0x7fe6380d1340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe630009e50 tx=0x7fe63002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90ac8b8a1d7cd8a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.031+0000 7fe6367fc640 1 -- 172.21.15.7:0/2960770867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630036070 con 0x7fe6380b5c40 2024-09-15T23:37:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.031+0000 7fe6367fc640 1 -- 172.21.15.7:0/2960770867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe630031600 con 0x7fe6380b5c40 2024-09-15T23:37:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.031+0000 7fe6367fc640 1 -- 172.21.15.7:0/2960770867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6300319d0 con 0x7fe6380b5c40 2024-09-15T23:37:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.032+0000 7fe63e227640 1 -- 172.21.15.7:0/2960770867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6380b5c40 msgr2=0x7fe6380d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.032+0000 7fe63e227640 1 --2- 172.21.15.7:0/2960770867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6380b5c40 0x7fe6380d1340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe630009e50 tx=0x7fe63002a710 comp rx=0 tx=0).stop 2024-09-15T23:37:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.032+0000 7fe63e227640 1 -- 172.21.15.7:0/2960770867 shutdown_connections 2024-09-15T23:37:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.032+0000 7fe63e227640 1 --2- 172.21.15.7:0/2960770867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6380b5c40 0x7fe6380d1340 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.032+0000 7fe63e227640 1 --2- 172.21.15.7:0/2960770867 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe638161f20 0x7fe6380d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.032+0000 7fe63e227640 1 -- 172.21.15.7:0/2960770867 >> 172.21.15.7:0/2960770867 conn(0x7fe63808bfd0 msgr2=0x7fe6380d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.032+0000 7fe63e227640 1 -- 172.21.15.7:0/2960770867 shutdown_connections 2024-09-15T23:37:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.032+0000 7fe63e227640 1 -- 172.21.15.7:0/2960770867 wait complete. 2024-09-15T23:37:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.033+0000 7fe63e227640 1 Processor -- start 2024-09-15T23:37:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.033+0000 7fe63e227640 1 -- start start 2024-09-15T23:37:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe63e227640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6380b5570 0x7fe6380b9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe63e227640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6380b5c40 0x7fe63815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe63e227640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe638161f20 0x7fe6380b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe63e227640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6380b38f0 con 0x7fe638161f20 2024-09-15T23:37:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe63e227640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe6380d0270 con 0x7fe6380b5570 2024-09-15T23:37:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe63e227640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe6380b4c90 con 0x7fe6380b5c40 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe637fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe638161f20 0x7fe6380b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe637fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe638161f20 0x7fe6380b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54622/0 (socket says 172.21.15.7:54622) 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe637fff640 1 -- 172.21.15.7:0/1895759018 learned_addr learned my addr 172.21.15.7:0/1895759018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe6377fe640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6380b5570 0x7fe6380b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe636ffd640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6380b5c40 0x7fe63815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe637fff640 1 -- 172.21.15.7:0/1895759018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6380b5c40 msgr2=0x7fe63815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe637fff640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6380b5c40 0x7fe63815b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe637fff640 1 -- 172.21.15.7:0/1895759018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6380b5570 msgr2=0x7fe6380b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe637fff640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6380b5570 0x7fe6380b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe637fff640 1 -- 172.21.15.7:0/1895759018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380b9db0 con 0x7fe638161f20 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe636ffd640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6380b5c40 0x7fe63815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.034+0000 7fe637fff640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe638161f20 0x7fe6380b26f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe630034040 tx=0x7fe630004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.035+0000 7fe61ffff640 1 -- 172.21.15.7:0/1895759018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63003f020 con 0x7fe638161f20 2024-09-15T23:37:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.035+0000 7fe61ffff640 1 -- 172.21.15.7:0/1895759018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe630037040 con 0x7fe638161f20 2024-09-15T23:37:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.035+0000 7fe63e227640 1 -- 172.21.15.7:0/1895759018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe638159b50 con 0x7fe638161f20 2024-09-15T23:37:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.035+0000 7fe61ffff640 1 -- 172.21.15.7:0/1895759018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630036070 con 0x7fe638161f20 2024-09-15T23:37:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.035+0000 7fe63e227640 1 -- 172.21.15.7:0/1895759018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63815a110 con 0x7fe638161f20 2024-09-15T23:37:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.036+0000 7fe61ffff640 1 -- 172.21.15.7:0/1895759018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe630042020 con 0x7fe638161f20 2024-09-15T23:37:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.037+0000 7fe61ffff640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe62406c2b0 0x7fe62406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.037+0000 7fe61ffff640 1 -- 172.21.15.7:0/1895759018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe6300aba10 con 0x7fe638161f20 2024-09-15T23:37:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.037+0000 7fe6377fe640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe62406c2b0 0x7fe62406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.037+0000 7fe6377fe640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe62406c2b0 0x7fe62406e770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe628009a30 tx=0x7fe628008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.038+0000 7fe640fc5740 1 Processor -- start 2024-09-15T23:37:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.038+0000 7fe640fc5740 1 -- start start 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfaa80 0x5627a6dfd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6df78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe640fc5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5627a6d5f520 con 0x5627a6dfd2c0 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe640fc5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5627a6d073e0 con 0x5627a6dfaa80 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe640fc5740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x5627a6d5b4e0 con 0x5627a6dfea50 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54628/0 (socket says 172.21.15.7:54628) 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe6377fe640 1 -- 172.21.15.7:0/1164036307 learned_addr learned my addr 172.21.15.7:0/1164036307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe637fff640 1 --2- 172.21.15.7:0/1164036307 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6df78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe636ffd640 1 --2- 172.21.15.7:0/1164036307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfaa80 0x5627a6dfd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe6377fe640 1 -- 172.21.15.7:0/1164036307 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 msgr2=0x5627a6df78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe6377fe640 1 --2- 172.21.15.7:0/1164036307 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6df78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe6377fe640 1 -- 172.21.15.7:0/1164036307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfaa80 msgr2=0x5627a6dfd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe6377fe640 1 --2- 172.21.15.7:0/1164036307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfaa80 0x5627a6dfd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe6377fe640 1 -- 172.21.15.7:0/1164036307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a6d222f0 con 0x5627a6dfd2c0 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe6377fe640 1 --2- 172.21.15.7:0/1164036307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42a60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe628011d20 tx=0x7fe628014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513b3f504c87d8b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe61d7fa640 1 -- 172.21.15.7:0/1164036307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628011d50 con 0x5627a6dfd2c0 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.039+0000 7fe61d7fa640 1 -- 172.21.15.7:0/1164036307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe628026070 con 0x5627a6dfd2c0 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe61d7fa640 1 -- 172.21.15.7:0/1164036307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628021640 con 0x5627a6dfd2c0 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe640fc5740 1 -- 172.21.15.7:0/1164036307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 msgr2=0x5627a6e42a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe640fc5740 1 --2- 172.21.15.7:0/1164036307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42a60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe628011d20 tx=0x7fe628014000 comp rx=0 tx=0).stop 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe640fc5740 1 -- 172.21.15.7:0/1164036307 shutdown_connections 2024-09-15T23:37:25.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe640fc5740 1 --2- 172.21.15.7:0/1164036307 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6df78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe640fc5740 1 --2- 172.21.15.7:0/1164036307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfaa80 0x5627a6dfd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe640fc5740 1 --2- 172.21.15.7:0/1164036307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42a60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe640fc5740 1 -- 172.21.15.7:0/1164036307 >> 172.21.15.7:0/1164036307 conn(0x5627a6df9a80 msgr2=0x5627a6e41f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe640fc5740 1 -- 172.21.15.7:0/1164036307 shutdown_connections 2024-09-15T23:37:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.040+0000 7fe640fc5740 1 -- 172.21.15.7:0/1164036307 wait complete. 2024-09-15T23:37:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.041+0000 7fe640fc5740 1 -- start start 2024-09-15T23:37:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.041+0000 7fe640fc5740 1 Processor -- start 2024-09-15T23:37:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfd2c0 0x5627a6e56720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfea50 0x5627a6dfbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfc320 0x5627a6e3b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe640fc5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5627a6df8f20 con 0x5627a6dfc320 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe640fc5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5627a6ce8450 con 0x5627a6dfea50 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe640fc5740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x5627a6e43290 con 0x5627a6dfd2c0 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe640fc5740 10 client.0 _ll_register_callbacks cb 0x5627a6a71ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe637fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfc320 0x5627a6e3b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe637fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfc320 0x5627a6e3b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54636/0 (socket says 172.21.15.7:54636) 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe637fff640 1 -- 172.21.15.7:0/3140306629 learned_addr learned my addr 172.21.15.7:0/3140306629 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe6377fe640 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfd2c0 0x5627a6e56720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe636ffd640 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfea50 0x5627a6dfbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe637fff640 1 -- 172.21.15.7:0/3140306629 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfd2c0 msgr2=0x5627a6e56720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe637fff640 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfd2c0 0x5627a6e56720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe637fff640 1 -- 172.21.15.7:0/3140306629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfea50 msgr2=0x5627a6dfbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe637fff640 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfea50 0x5627a6dfbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe637fff640 1 -- 172.21.15.7:0/3140306629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a6a722d0 con 0x5627a6dfc320 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe637fff640 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfc320 0x5627a6e3b820 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe630075e00 tx=0x7fe6300095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dbda375238d936 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe6167fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe6167fc640 1 -- 172.21.15.7:0/3140306629 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63007dca0 con 0x5627a6dfc320 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe6167fc640 1 -- 172.21.15.7:0/3140306629 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe6300800b0 con 0x5627a6dfc320 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.042+0000 7fe640fc5740 1 -- 172.21.15.7:0/3140306629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe628026070 con 0x5627a6dfc320 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe6167fc640 1 -- 172.21.15.7:0/3140306629 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fe6300824d0 con 0x5627a6dfc320 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe640fc5740 10 client.5028 fetch_fsmap learned FSMap version 15 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe640fc5740 1 -- 172.21.15.7:0/3140306629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x5627a6b3c6b0 con 0x5627a6dfc320 2024-09-15T23:37:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe6167fc640 1 -- 172.21.15.7:0/3140306629 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe628026070 con 0x5627a6dfc320 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe6167fc640 1 -- 172.21.15.7:0/3140306629 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 15) ==== 3502+0+0 (secure 0 0 0) 0x7fe630091b50 con 0x5627a6dfc320 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe640fc5740 10 client.5028 fetch_fsmap finished waiting for FSMap version 15 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe640fc5740 10 client.5028 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe640fc5740 20 client.5028 populate_metadata read hostname 'smithi007' 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] conn(0x5627a6e1c5c0 0x5627a6e26cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe640fc5740 4 client.5028 mds_command: new command op to 4830 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe640fc5740 1 -- 172.21.15.7:0/3140306629 --> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627a6d6cc50 con 0x5627a6e1c5c0 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe6377fe640 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] conn(0x5627a6e1c5c0 0x5627a6e26cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe6377fe640 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] conn(0x5627a6e1c5c0 0x5627a6e26cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dee08e9355fd96a server_cookie=863e09080c90098e in_seq=0 out_seq=0 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.043+0000 7fe6167fc640 10 client.5028 ms_handle_connect on v2:172.21.15.7:6835/1729414280 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe6167fc640 1 -- 172.21.15.7:0/3140306629 <== mds.0 v2:172.21.15.7:6835/1729414280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627a6d6cc50 con 0x5627a6e1c5c0 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe6167fc640 10 client.5028 handle_command_reply: tid=0 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe640fc5740 1 client.5028 shutdown 2024-09-15T23:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe640fc5740 1 -- 172.21.15.7:0/3140306629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfc320 msgr2=0x5627a6e3b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfc320 0x5627a6e3b820 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe630075e00 tx=0x7fe6300095d0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe640fc5740 1 -- 172.21.15.7:0/3140306629 shutdown_connections 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] conn(0x5627a6e1c5c0 0x5627a6e26cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfc320 0x5627a6e3b820 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfea50 0x5627a6dfbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe6167fc640 0 client.5028 ms_handle_reset on v2:172.21.15.7:6835/1729414280 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3140306629 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfd2c0 0x5627a6e56720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.044+0000 7fe640fc5740 1 -- 172.21.15.7:0/3140306629 >> 172.21.15.7:0/3140306629 conn(0x5627a6dfaa80 msgr2=0x5627a6e21f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.045+0000 7fe640fc5740 1 -- 172.21.15.7:0/3140306629 shutdown_connections 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.045+0000 7fe640fc5740 1 -- 172.21.15.7:0/3140306629 wait complete. 2024-09-15T23:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.045+0000 7fe640fc5740 20 client.5028 trim_cache size 0 max 16384 2024-09-15T23:37:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.063+0000 7fe640fc5740 1 Processor -- start 2024-09-15T23:37:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.063+0000 7fe640fc5740 1 -- start start 2024-09-15T23:37:25.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 0x5627a6e5c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e5c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6e34590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe640fc5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6382003a0 con 0x5627a6dfd2c0 2024-09-15T23:37:25.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe640fc5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe6381f5620 con 0x5627a6dfc320 2024-09-15T23:37:25.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe640fc5740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe6381f5c70 con 0x5627a6dfea50 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe636ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e5c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe636ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e5c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54642/0 (socket says 172.21.15.7:54642) 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe636ffd640 1 -- 172.21.15.7:0/3475206568 learned_addr learned my addr 172.21.15.7:0/3475206568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe636ffd640 1 -- 172.21.15.7:0/3475206568 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 msgr2=0x5627a6e34590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe636ffd640 1 --2- 172.21.15.7:0/3475206568 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6e34590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe6377fe640 1 --2- 172.21.15.7:0/3475206568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 0x5627a6e5c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe636ffd640 1 -- 172.21.15.7:0/3475206568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 msgr2=0x5627a6e5c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe636ffd640 1 --2- 172.21.15.7:0/3475206568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 0x5627a6e5c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe636ffd640 1 -- 172.21.15.7:0/3475206568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a6df8740 con 0x5627a6dfd2c0 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe636ffd640 1 --2- 172.21.15.7:0/3475206568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e5c870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0016c0 tx=0x7fe62c001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec74ba9a154a0d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe61cff9640 1 -- 172.21.15.7:0/3475206568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c012820 con 0x5627a6dfd2c0 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe61cff9640 1 -- 172.21.15.7:0/3475206568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c0129c0 con 0x5627a6dfd2c0 2024-09-15T23:37:25.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.064+0000 7fe61cff9640 1 -- 172.21.15.7:0/3475206568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c012d70 con 0x5627a6dfd2c0 2024-09-15T23:37:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.065+0000 7fe640fc5740 1 -- 172.21.15.7:0/3475206568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 msgr2=0x5627a6e5c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.065+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3475206568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e5c870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0016c0 tx=0x7fe62c001b80 comp rx=0 tx=0).stop 2024-09-15T23:37:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.065+0000 7fe640fc5740 1 -- 172.21.15.7:0/3475206568 shutdown_connections 2024-09-15T23:37:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.065+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3475206568 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6e34590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.065+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3475206568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e5c870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.065+0000 7fe640fc5740 1 --2- 172.21.15.7:0/3475206568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 0x5627a6e5c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.065+0000 7fe640fc5740 1 -- 172.21.15.7:0/3475206568 >> 172.21.15.7:0/3475206568 conn(0x5627a6dfaa80 msgr2=0x5627a6e3b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.065+0000 7fe640fc5740 1 -- 172.21.15.7:0/3475206568 shutdown_connections 2024-09-15T23:37:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.065+0000 7fe640fc5740 1 -- 172.21.15.7:0/3475206568 wait complete. 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.066+0000 7fe640fc5740 1 -- start start 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.066+0000 7fe640fc5740 1 Processor -- start 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.066+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 0x5627a6ebeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.066+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.066+0000 7fe640fc5740 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6e22350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.066+0000 7fe640fc5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5627a6e25510 con 0x5627a6dfd2c0 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.066+0000 7fe640fc5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5627a6e27cb0 con 0x5627a6dfc320 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.066+0000 7fe640fc5740 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x5627a6e1ca10 con 0x5627a6dfea50 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe640fc5740 10 client.0 _ll_register_callbacks cb 0x7fe6300795d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe636ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe636ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54658/0 (socket says 172.21.15.7:54658) 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe636ffd640 1 -- 172.21.15.7:0/2665284222 learned_addr learned my addr 172.21.15.7:0/2665284222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe6377fe640 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 0x5627a6ebeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe637fff640 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6e22350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe636ffd640 1 -- 172.21.15.7:0/2665284222 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 msgr2=0x5627a6e22350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe636ffd640 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6e22350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe636ffd640 1 -- 172.21.15.7:0/2665284222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 msgr2=0x5627a6ebeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe636ffd640 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 0x5627a6ebeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe636ffd640 1 -- 172.21.15.7:0/2665284222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a6e42740 con 0x5627a6dfd2c0 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe636ffd640 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe62c001bc0 tx=0x7fe62c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb57e56c81d29a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe6167fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe6167fc640 1 -- 172.21.15.7:0/2665284222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c012ea0 con 0x5627a6dfd2c0 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe6167fc640 1 -- 172.21.15.7:0/2665284222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe62c007460 con 0x5627a6dfd2c0 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.067+0000 7fe640fc5740 1 -- 172.21.15.7:0/2665284222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe62c0129c0 con 0x5627a6dfd2c0 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe6167fc640 1 -- 172.21.15.7:0/2665284222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fe62c032050 con 0x5627a6dfd2c0 2024-09-15T23:37:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe640fc5740 10 client.5034 fetch_fsmap learned FSMap version 15 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe640fc5740 1 -- 172.21.15.7:0/2665284222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x5627a6e454c0 con 0x5627a6dfd2c0 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe6167fc640 1 -- 172.21.15.7:0/2665284222 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c0129c0 con 0x5627a6dfd2c0 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe6167fc640 1 -- 172.21.15.7:0/2665284222 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 15) ==== 3502+0+0 (secure 0 0 0) 0x7fe62c035020 con 0x5627a6dfd2c0 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe640fc5740 10 client.5034 fetch_fsmap finished waiting for FSMap version 15 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe640fc5740 10 client.5034 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe640fc5740 20 client.5034 populate_metadata read hostname 'smithi007' 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe640fc5740 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] conn(0x5627a6e3e570 0x5627a6e3e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe640fc5740 4 client.5034 mds_command: new command op to 4830 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe640fc5740 1 -- 172.21.15.7:0/2665284222 --> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] -- command(tid 0: {"prefix": "status"}) -- 0x5627a6d66fa0 con 0x5627a6e3e570 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe6377fe640 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] conn(0x5627a6e3e570 0x5627a6e3e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe6377fe640 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] conn(0x5627a6e3e570 0x5627a6e3e950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36aac0b8adfb1bda server_cookie=25510d7fc356ceee in_seq=0 out_seq=0 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe6167fc640 10 client.5034 ms_handle_connect on v2:172.21.15.7:6835/1729414280 2024-09-15T23:37:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.068+0000 7fe6167fc640 1 -- 172.21.15.7:0/2665284222 <== mds.0 v2:172.21.15.7:6835/1729414280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5627a6d66fa0 con 0x5627a6e3e570 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe6167fc640 10 client.5034 handle_command_reply: tid=0 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 client.5034 shutdown 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 -- 172.21.15.7:0/2665284222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 msgr2=0x5627a6e42200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe62c001bc0 tx=0x7fe62c000bc0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 -- 172.21.15.7:0/2665284222 shutdown_connections 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.7:6835/1729414280,v1:172.21.15.7:6837/1729414280] conn(0x5627a6e3e570 0x5627a6e3e950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x5627a6dfea50 0x5627a6e22350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627a6dfd2c0 0x5627a6e42200 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe6167fc640 0 client.5034 ms_handle_reset on v2:172.21.15.7:6835/1729414280 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 --2- 172.21.15.7:0/2665284222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627a6dfc320 0x5627a6ebeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 -- 172.21.15.7:0/2665284222 >> 172.21.15.7:0/2665284222 conn(0x5627a6dfaa80 msgr2=0x5627a6df2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 -- 172.21.15.7:0/2665284222 shutdown_connections 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 1 -- 172.21.15.7:0/2665284222 wait complete. 2024-09-15T23:37:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe640fc5740 20 client.5034 trim_cache size 0 max 16384 2024-09-15T23:37:25.076 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:25.076 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9da25f4c-4673-468c-a40c-f01392dffc9b", 2024-09-15T23:37:25.076 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-15T23:37:25.076 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4830, 2024-09-15T23:37:25.076 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-15T23:37:25.076 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-15T23:37:25.076 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-15T23:37:25.076 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 9.0446033270000008, 2024-09-15T23:37:25.076 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 15, 2024-09-15T23:37:25.077 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 39, 2024-09-15T23:37:25.077 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 39, 2024-09-15T23:37:25.077 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 15.927558877999999 2024-09-15T23:37:25.077 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 -- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe62406c2b0 msgr2=0x7fe62406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe62406c2b0 0x7fe62406e770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe628009a30 tx=0x7fe628008040 comp rx=0 tx=0).stop 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 -- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe638161f20 msgr2=0x7fe6380b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe638161f20 0x7fe6380b26f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe630034040 tx=0x7fe630004a40 comp rx=0 tx=0).stop 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 -- 172.21.15.7:0/1895759018 shutdown_connections 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe638161f20 0x7fe6380b26f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fe62406c2b0 0x7fe62406e770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6380b5c40 0x7fe63815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 --2- 172.21.15.7:0/1895759018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6380b5570 0x7fe6380b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.069+0000 7fe63e227640 1 -- 172.21.15.7:0/1895759018 >> 172.21.15.7:0/1895759018 conn(0x7fe63808bfd0 msgr2=0x7fe638161200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.070+0000 7fe63e227640 1 -- 172.21.15.7:0/1895759018 shutdown_connections 2024-09-15T23:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.070+0000 7fe63e227640 1 -- 172.21.15.7:0/1895759018 wait complete. 2024-09-15T23:37:25.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-15T23:37:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.183+0000 7f30d82af640 1 Processor -- start 2024-09-15T23:37:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.183+0000 7f30d82af640 1 -- start start 2024-09-15T23:37:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d82af640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30d015e6d0 0x7f30d00d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d82af640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f30d00d51c0 0x7f30d00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d82af640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d00b3d10 con 0x7f30d00d30d0 2024-09-15T23:37:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d82af640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f30d00d3500 con 0x7f30d015e6d0 2024-09-15T23:37:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d82af640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f30d00cfcd0 con 0x7f30d00d51c0 2024-09-15T23:37:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d604b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30d015e6d0 0x7f30d00d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d604b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30d015e6d0 0x7f30d00d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54674/0 (socket says 172.21.15.7:54674) 2024-09-15T23:37:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d604b640 1 -- 172.21.15.7:0/1277968414 learned_addr learned my addr 172.21.15.7:0/1277968414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d584a640 1 --2- 172.21.15.7:0/1277968414 >> v2:172.21.15.18:3300/0 conn(0x7f30d00d51c0 0x7f30d00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d604b640 1 -- 172.21.15.7:0/1277968414 >> v2:172.21.15.18:3300/0 conn(0x7f30d00d51c0 msgr2=0x7f30d00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d604b640 1 --2- 172.21.15.7:0/1277968414 >> v2:172.21.15.18:3300/0 conn(0x7f30d00d51c0 0x7f30d00d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d604b640 1 -- 172.21.15.7:0/1277968414 >> v1:172.21.15.18:6789/0 conn(0x7f30d00d30d0 legacy=0x7f30d00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.184+0000 7f30d604b640 1 -- 172.21.15.7:0/1277968414 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d00cff50 con 0x7f30d015e6d0 2024-09-15T23:37:25.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.185+0000 7f30d604b640 1 --2- 172.21.15.7:0/1277968414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e6d0 0x7f30d00d21b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f30d008c600 tx=0x7f30cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d26be6fc0ccaff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.185+0000 7f30d5049640 1 -- 172.21.15.7:0/1277968414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc037070 con 0x7f30d015e6d0 2024-09-15T23:37:25.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.185+0000 7f30d5049640 1 -- 172.21.15.7:0/1277968414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30cc032440 con 0x7f30d015e6d0 2024-09-15T23:37:25.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.185+0000 7f30d5049640 1 -- 172.21.15.7:0/1277968414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc032810 con 0x7f30d015e6d0 2024-09-15T23:37:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.186+0000 7f30d82af640 1 -- 172.21.15.7:0/1277968414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e6d0 msgr2=0x7f30d00d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.186+0000 7f30d82af640 1 --2- 172.21.15.7:0/1277968414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e6d0 0x7f30d00d21b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f30d008c600 tx=0x7f30cc02a230 comp rx=0 tx=0).stop 2024-09-15T23:37:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.186+0000 7f30d82af640 1 -- 172.21.15.7:0/1277968414 shutdown_connections 2024-09-15T23:37:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.186+0000 7f30d82af640 1 --2- 172.21.15.7:0/1277968414 >> v2:172.21.15.18:3300/0 conn(0x7f30d00d51c0 0x7f30d00d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.186+0000 7f30d82af640 1 --2- 172.21.15.7:0/1277968414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e6d0 0x7f30d00d21b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.186+0000 7f30d82af640 1 -- 172.21.15.7:0/1277968414 >> 172.21.15.7:0/1277968414 conn(0x7f30d008bfd0 msgr2=0x7f30d00d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.186+0000 7f30d82af640 1 -- 172.21.15.7:0/1277968414 shutdown_connections 2024-09-15T23:37:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.186+0000 7f30d82af640 1 -- 172.21.15.7:0/1277968414 wait complete. 2024-09-15T23:37:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.187+0000 7f30d82af640 1 Processor -- start 2024-09-15T23:37:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.187+0000 7f30d82af640 1 -- start start 2024-09-15T23:37:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d82af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d00d30d0 0x7f30d0155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d82af640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30d00d51c0 0x7f30d00b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d82af640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e6d0 0x7f30d00b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d82af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30d00d0510 con 0x7f30d00d30d0 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d82af640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f30d015eab0 con 0x7f30d00d51c0 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d82af640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f30d00d26f0 con 0x7f30d015e6d0 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d604b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d00d30d0 0x7f30d0155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d604b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d00d30d0 0x7f30d0155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54678/0 (socket says 172.21.15.7:54678) 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d604b640 1 -- 172.21.15.7:0/1497699556 learned_addr learned my addr 172.21.15.7:0/1497699556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d684c640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e6d0 0x7f30d00b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d584a640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30d00d51c0 0x7f30d00b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d604b640 1 -- 172.21.15.7:0/1497699556 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e6d0 msgr2=0x7f30d00b9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d604b640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e6d0 0x7f30d00b9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d604b640 1 -- 172.21.15.7:0/1497699556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30d00d51c0 msgr2=0x7f30d00b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d604b640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30d00d51c0 0x7f30d00b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d604b640 1 -- 172.21.15.7:0/1497699556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d00b76f0 con 0x7f30d00d30d0 2024-09-15T23:37:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d684c640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e6d0 0x7f30d00b9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.188+0000 7f30d604b640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d00d30d0 0x7f30d0155c20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f30cc000c00 tx=0x7f30cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.189+0000 7f30c6ffd640 1 -- 172.21.15.7:0/1497699556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc041020 con 0x7f30d00d30d0 2024-09-15T23:37:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.189+0000 7f30c6ffd640 1 -- 172.21.15.7:0/1497699556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30cc0306c0 con 0x7f30d00d30d0 2024-09-15T23:37:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.189+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d00cd560 con 0x7f30d00d30d0 2024-09-15T23:37:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.189+0000 7f30c6ffd640 1 -- 172.21.15.7:0/1497699556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc037040 con 0x7f30d00d30d0 2024-09-15T23:37:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.189+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d00cde00 con 0x7f30d00d30d0 2024-09-15T23:37:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.190+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d006ec50 con 0x7f30d00d30d0 2024-09-15T23:37:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.190+0000 7f30c6ffd640 1 -- 172.21.15.7:0/1497699556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f30cc044020 con 0x7f30d00d30d0 2024-09-15T23:37:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.191+0000 7f30c6ffd640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f30b406c2b0 0x7f30b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.191+0000 7f30c6ffd640 1 -- 172.21.15.7:0/1497699556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f30cc0abe80 con 0x7f30d00d30d0 2024-09-15T23:37:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.191+0000 7f30d584a640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f30b406c2b0 0x7f30b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.191+0000 7f30d584a640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f30b406c2b0 0x7f30b406e770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30d00ce6c0 tx=0x7f30c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:25.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.195+0000 7f30c6ffd640 1 -- 172.21.15.7:0/1497699556 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f30d006ec50 con 0x7f30d00d30d0 2024-09-15T23:37:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.405+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f30d00d2b90 con 0x7f30d00d30d0 2024-09-15T23:37:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.405+0000 7f30c6ffd640 1 -- 172.21.15.7:0/1497699556 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7f30cc04b030 con 0x7f30d00d30d0 2024-09-15T23:37:25.406 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-15T23:37:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f30b406c2b0 msgr2=0x7f30b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f30b406c2b0 0x7f30b406e770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30d00ce6c0 tx=0x7f30c0008040 comp rx=0 tx=0).stop 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d00d30d0 msgr2=0x7f30d0155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d00d30d0 0x7f30d0155c20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f30cc000c00 tx=0x7f30cc0046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 shutdown_connections 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f30b406c2b0 0x7f30b406e770 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e6d0 0x7f30d00b9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30d00d51c0 0x7f30d00b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 --2- 172.21.15.7:0/1497699556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d00d30d0 0x7f30d0155c20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 >> 172.21.15.7:0/1497699556 conn(0x7f30d008bfd0 msgr2=0x7f30d00d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.407+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 shutdown_connections 2024-09-15T23:37:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25.408+0000 7f30d82af640 1 -- 172.21.15.7:0/1497699556 wait complete. 2024-09-15T23:37:25.420 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-15T23:37:25.420 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T23:37:25.420 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-15T23:37:25.421 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-15T23:37:25.421 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T23:37:25.421 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-15T23:37:25.421 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-15T23:37:25.421 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T23:37:25.421 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-15T23:37:25.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:25.422 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-15T23:37:25.439 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T23:37:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T23:37:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T23:37:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:37:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-15T23:37:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:37:25.441 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:37:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-15T23:37:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-15T23:37:25.441 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:37:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-15T23:37:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-15T23:37:25.442 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T23:37:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-15T23:37:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-15T23:37:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T23:37:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42710sec preferred_lft 42710sec 2024-09-15T23:37:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-15T23:37:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:37:25.443 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T23:37:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-15T23:37:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-15T23:37:25.444 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-15T23:37:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 16:de:de:50:bf:62 brd ff:ff:ff:ff:ff:ff 2024-09-15T23:37:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-15T23:37:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:37:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::a0db:6aff:fe42:a0c6/64 scope link 2024-09-15T23:37:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:37:25.445 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-15T23:37:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 5a:f1:a0:f2:08:e7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T23:37:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::58f1:a0ff:fef2:8e7/64 scope link 2024-09-15T23:37:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:37:25.445 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-15T23:37:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 16:de:de:50:bf:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-15T23:37:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::14de:deff:fe50:bf62/64 scope link 2024-09-15T23:37:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-15T23:37:25.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:25.447 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T23:37:25.504 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-15T23:37:25.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:25.505 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-15T23:37:25.562 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-15T23:37:25.562 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T23:37:25.562 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T23:37:25.562 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T23:37:25.562 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T23:37:25.563 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T23:37:25.563 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-15T23:37:25.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T23:37:25.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T23:37:25.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T23:37:25.563 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-15T23:37:25.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:37:25.648 DEBUG:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:37:25.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T23:37:25.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:37:25.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:25.691 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-15T23:37:25.747 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-15T23:37:25.748 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-15T23:37:25.748 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-15T23:37:25.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:25.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-15T23:37:25.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:37:25.847 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-15T23:37:25.902 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-15T23:37:25.903 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T23:37:25.903 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-15T23:37:25.958 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-15T23:37:25.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:25.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-09-15T23:37:26.043 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-09-15T23:37:26.044 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-09-15T23:37:26.044 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-15T23:37:26.053 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-09-15T23:37:26.053 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-09-15T23:37:26.053 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-15T23:37:26.053 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:37:26.054 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:37:26.059 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-09-15T23:37:26.059 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-09-15T23:37:26.059 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-15T23:37:26.059 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.18:3300,172.21.15.18:3301" 2024-09-15T23:37:26.059 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-15T23:37:26.059 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790" 2024-09-15T23:37:26.059 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.18:6789,172.21.15.18:6790:/ 2024-09-15T23:37:26.059 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs,key=0,fsid=9da25f4c-4673-468c-a40c-f01392dffc9b" will pass to kernel 2024-09-15T23:37:26.060 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-15T23:37:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-15T23:37:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-15T23:37:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-15T23:37:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-15T23:37:26.061 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T23:37:26.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:26.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-15T23:37:26.087 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T23:37:26.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T23:37:26.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T23:37:26.192 INFO:teuthology.orchestra.run.smithi007.stdout:client4700 2024-09-15T23:37:26.192 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:26.192 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:26.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-15T23:37:26.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.351+0000 7faefca42640 1 Processor -- start 2024-09-15T23:37:26.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.351+0000 7faefca42640 1 -- start start 2024-09-15T23:37:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.352+0000 7faefca42640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7faef81558e0 0x7faef80ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.352+0000 7faefca42640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faef806e060 0x7faef815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.352+0000 7faefca42640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef815f850 con 0x7faef815f430 2024-09-15T23:37:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.352+0000 7faefca42640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7faef80b3160 con 0x7faef806e060 2024-09-15T23:37:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.352+0000 7faefca42640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7faef80af550 con 0x7faef81558e0 2024-09-15T23:37:26.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.352+0000 7faef5d9b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faef806e060 0x7faef815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.352+0000 7faef5d9b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faef806e060 0x7faef815eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54682/0 (socket says 172.21.15.7:54682) 2024-09-15T23:37:26.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.352+0000 7faef5d9b640 1 -- 172.21.15.7:0/230253939 learned_addr learned my addr 172.21.15.7:0/230253939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:26.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.353+0000 7faef659c640 1 --2- 172.21.15.7:0/230253939 >> v2:172.21.15.18:3300/0 conn(0x7faef81558e0 0x7faef80ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.353+0000 7faef5d9b640 1 -- 172.21.15.7:0/230253939 >> v2:172.21.15.18:3300/0 conn(0x7faef81558e0 msgr2=0x7faef80ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.353+0000 7faef5d9b640 1 --2- 172.21.15.7:0/230253939 >> v2:172.21.15.18:3300/0 conn(0x7faef81558e0 0x7faef80ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.353+0000 7faef5d9b640 1 -- 172.21.15.7:0/230253939 >> v1:172.21.15.7:6789/0 conn(0x7faef815f430 legacy=0x7faef815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.353+0000 7faef5d9b640 1 -- 172.21.15.7:0/230253939 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef80af270 con 0x7faef806e060 2024-09-15T23:37:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.353+0000 7faef559a640 1 -- 172.21.15.7:0/230253939 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2377320395 0 0) 0x7faef815f850 con 0x7faef815f430 2024-09-15T23:37:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faef5d9b640 1 --2- 172.21.15.7:0/230253939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 0x7faef815eef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faef806db80 tx=0x7faeec02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c873cfcd35f187 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faef559a640 1 -- 172.21.15.7:0/230253939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec037070 con 0x7faef806e060 2024-09-15T23:37:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faef559a640 1 -- 172.21.15.7:0/230253939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec032440 con 0x7faef806e060 2024-09-15T23:37:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faef559a640 1 -- 172.21.15.7:0/230253939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec032810 con 0x7faef806e060 2024-09-15T23:37:26.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faefca42640 1 -- 172.21.15.7:0/230253939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 msgr2=0x7faef815eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faefca42640 1 --2- 172.21.15.7:0/230253939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 0x7faef815eef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faef806db80 tx=0x7faeec02a150 comp rx=0 tx=0).stop 2024-09-15T23:37:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faefca42640 1 -- 172.21.15.7:0/230253939 shutdown_connections 2024-09-15T23:37:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faefca42640 1 --2- 172.21.15.7:0/230253939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 0x7faef815eef0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faefca42640 1 --2- 172.21.15.7:0/230253939 >> v2:172.21.15.18:3300/0 conn(0x7faef81558e0 0x7faef80ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faefca42640 1 -- 172.21.15.7:0/230253939 >> 172.21.15.7:0/230253939 conn(0x7faef80b1e60 msgr2=0x7faef80b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faefca42640 1 -- 172.21.15.7:0/230253939 shutdown_connections 2024-09-15T23:37:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.354+0000 7faefca42640 1 -- 172.21.15.7:0/230253939 wait complete. 2024-09-15T23:37:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.356+0000 7faefca42640 1 Processor -- start 2024-09-15T23:37:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.356+0000 7faefca42640 1 -- start start 2024-09-15T23:37:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.356+0000 7faefca42640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 0x7faef81f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.356+0000 7faefca42640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faef81558e0 0x7faef8154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.356+0000 7faefca42640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faef815f430 0x7faef80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.356+0000 7faefca42640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faef80aeef0 con 0x7faef806e060 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.356+0000 7faefca42640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faef8072300 con 0x7faef81558e0 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.356+0000 7faefca42640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7faef8155cc0 con 0x7faef815f430 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 0x7faef81f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 0x7faef81f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54688/0 (socket says 172.21.15.7:54688) 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef659c640 1 -- 172.21.15.7:0/924901225 learned_addr learned my addr 172.21.15.7:0/924901225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef5d9b640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faef81558e0 0x7faef8154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef6d9d640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faef815f430 0x7faef80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef659c640 1 -- 172.21.15.7:0/924901225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faef815f430 msgr2=0x7faef80b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef659c640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faef815f430 0x7faef80b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef659c640 1 -- 172.21.15.7:0/924901225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faef81558e0 msgr2=0x7faef8154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef659c640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faef81558e0 0x7faef8154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef659c640 1 -- 172.21.15.7:0/924901225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef8154060 con 0x7faef806e060 2024-09-15T23:37:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faef659c640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 0x7faef81f3a20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9b0 tx=0x7faee800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faee6ffd640 1 -- 172.21.15.7:0/924901225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8017070 con 0x7faef806e060 2024-09-15T23:37:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faee6ffd640 1 -- 172.21.15.7:0/924901225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee800c9a0 con 0x7faef806e060 2024-09-15T23:37:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef8153200 con 0x7faef806e060 2024-09-15T23:37:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faee6ffd640 1 -- 172.21.15.7:0/924901225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee800cd90 con 0x7faef806e060 2024-09-15T23:37:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.357+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81537c0 con 0x7faef806e060 2024-09-15T23:37:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.358+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef815eef0 con 0x7faef806e060 2024-09-15T23:37:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.359+0000 7faee6ffd640 1 -- 172.21.15.7:0/924901225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faee801a070 con 0x7faef806e060 2024-09-15T23:37:26.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.359+0000 7faee6ffd640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7faed806c2b0 0x7faed806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.359+0000 7faee6ffd640 1 -- 172.21.15.7:0/924901225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7faee808b4b0 con 0x7faef806e060 2024-09-15T23:37:26.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.359+0000 7faef5d9b640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7faed806c2b0 0x7faed806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.360+0000 7faef5d9b640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7faed806c2b0 0x7faed806e770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faeec034000 tx=0x7faeec02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.363+0000 7faee6ffd640 1 -- 172.21.15.7:0/924901225 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faee80594d0 con 0x7faef806e060 2024-09-15T23:37:26.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.561+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7faef8155100 con 0x7faed806c2b0 2024-09-15T23:37:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.562+0000 7faee6ffd640 1 -- 172.21.15.7:0/924901225 <== mgr.4104 v2:172.21.15.7:6832/3993537392 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7faef8155100 con 0x7faed806c2b0 2024-09-15T23:37:26.563 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-15T23:37:26.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-15T23:37:26.563 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephfs" 2024-09-15T23:37:26.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-15T23:37:26.563 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-15T23:37:26.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.564+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7faed806c2b0 msgr2=0x7faed806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.564+0000 7faefca42640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7faed806c2b0 0x7faed806e770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faeec034000 tx=0x7faeec02e000 comp rx=0 tx=0).stop 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 msgr2=0x7faef81f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 0x7faef81f3a20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9b0 tx=0x7faee800ee70 comp rx=0 tx=0).stop 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 shutdown_connections 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7faed806c2b0 0x7faed806e770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faef815f430 0x7faef80b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faef81558e0 0x7faef8154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 --2- 172.21.15.7:0/924901225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef806e060 0x7faef81f3a20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 >> 172.21.15.7:0/924901225 conn(0x7faef80b1e60 msgr2=0x7faef815d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 shutdown_connections 2024-09-15T23:37:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.565+0000 7faefca42640 1 -- 172.21.15.7:0/924901225 wait complete. 2024-09-15T23:37:26.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-09-15T23:37:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.681+0000 7fa17af86640 1 Processor -- start 2024-09-15T23:37:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.681+0000 7fa17af86640 1 -- start start 2024-09-15T23:37:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa17af86640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa174161f20 0x7fa1740d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa17af86640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1740d3550 con 0x7fa1740b5570 2024-09-15T23:37:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa17af86640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1740d4f80 con 0x7fa1740b5c40 2024-09-15T23:37:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa17af86640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa1740d5540 con 0x7fa174161f20 2024-09-15T23:37:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa173fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa1740b5570 0x7fa1740d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51986/0 (socket says 172.21.15.7:51986) 2024-09-15T23:37:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa173fff640 1 -- 172.21.15.7:0/3832318495 learned_addr learned my addr 172.21.15.7:0/3832318495 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa178d22640 1 --2- 172.21.15.7:0/3832318495 >> v2:172.21.15.18:3300/0 conn(0x7fa174161f20 0x7fa1740d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa1737fe640 1 -- 172.21.15.7:0/3832318495 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894705405 0 0) 0x7fa1740d3550 con 0x7fa1740b5570 2024-09-15T23:37:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa1737fe640 1 -- 172.21.15.7:0/3832318495 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa15c003520 con 0x7fa1740b5570 2024-09-15T23:37:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa178d22640 1 -- 172.21.15.7:0/3832318495 >> v1:172.21.15.18:6789/0 conn(0x7fa1740b5c40 legacy=0x7fa1740d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa178d22640 1 -- 172.21.15.7:0/3832318495 >> v1:172.21.15.7:6789/0 conn(0x7fa1740b5570 legacy=0x7fa1740d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.682+0000 7fa178d22640 1 -- 172.21.15.7:0/3832318495 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1740d5260 con 0x7fa174161f20 2024-09-15T23:37:26.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.683+0000 7fa178d22640 1 --2- 172.21.15.7:0/3832318495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 0x7fa1740d0e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa1640097a0 tx=0x7fa16402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9afc90838af77b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:26.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.683+0000 7fa1737fe640 1 -- 172.21.15.7:0/3832318495 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164037070 con 0x7fa174161f20 2024-09-15T23:37:26.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.683+0000 7fa1737fe640 1 -- 172.21.15.7:0/3832318495 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa164032440 con 0x7fa174161f20 2024-09-15T23:37:26.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.683+0000 7fa1737fe640 1 -- 172.21.15.7:0/3832318495 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1640327f0 con 0x7fa174161f20 2024-09-15T23:37:26.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.684+0000 7fa17af86640 1 -- 172.21.15.7:0/3832318495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 msgr2=0x7fa1740d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.684+0000 7fa17af86640 1 --2- 172.21.15.7:0/3832318495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 0x7fa1740d0e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa1640097a0 tx=0x7fa16402a230 comp rx=0 tx=0).stop 2024-09-15T23:37:26.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.684+0000 7fa17af86640 1 -- 172.21.15.7:0/3832318495 shutdown_connections 2024-09-15T23:37:26.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.684+0000 7fa17af86640 1 --2- 172.21.15.7:0/3832318495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 0x7fa1740d0e00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.684+0000 7fa17af86640 1 -- 172.21.15.7:0/3832318495 >> 172.21.15.7:0/3832318495 conn(0x7fa17408bfd0 msgr2=0x7fa1740d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:26.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.684+0000 7fa17af86640 1 -- 172.21.15.7:0/3832318495 shutdown_connections 2024-09-15T23:37:26.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.684+0000 7fa17af86640 1 -- 172.21.15.7:0/3832318495 wait complete. 2024-09-15T23:37:26.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.685+0000 7fa17af86640 1 Processor -- start 2024-09-15T23:37:26.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.685+0000 7fa17af86640 1 -- start start 2024-09-15T23:37:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa17af86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1740b5570 0x7fa17415a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa17af86640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1740b5c40 0x7fa17414dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa17af86640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 0x7fa17415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa17af86640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740d0270 con 0x7fa1740b5570 2024-09-15T23:37:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa17af86640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740d4d30 con 0x7fa174161f20 2024-09-15T23:37:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa17af86640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa1740b38f0 con 0x7fa1740b5c40 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa178d22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1740b5570 0x7fa17415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa178d22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1740b5570 0x7fa17415a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54690/0 (socket says 172.21.15.7:54690) 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa178d22640 1 -- 172.21.15.7:0/1839966772 learned_addr learned my addr 172.21.15.7:0/1839966772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa179523640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 0x7fa17415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.686+0000 7fa173fff640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1740b5c40 0x7fa17414dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa178d22640 1 -- 172.21.15.7:0/1839966772 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1740b5c40 msgr2=0x7fa17414dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa178d22640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1740b5c40 0x7fa17414dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa178d22640 1 -- 172.21.15.7:0/1839966772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 msgr2=0x7fa17415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa178d22640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 0x7fa17415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa178d22640 1 -- 172.21.15.7:0/1839966772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17415ad60 con 0x7fa1740b5570 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa179523640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 0x7fa17415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa178d22640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1740b5570 0x7fa17415a780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa1640097a0 tx=0x7fa1640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa1717fa640 1 -- 172.21.15.7:0/1839966772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164037050 con 0x7fa1740b5570 2024-09-15T23:37:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa1717fa640 1 -- 172.21.15.7:0/1839966772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa164038040 con 0x7fa1740b5570 2024-09-15T23:37:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa1717fa640 1 -- 172.21.15.7:0/1839966772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164004470 con 0x7fa1740b5570 2024-09-15T23:37:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17415b040 con 0x7fa1740b5570 2024-09-15T23:37:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.687+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17415b600 con 0x7fa1740b5570 2024-09-15T23:37:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.688+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1740d0e00 con 0x7fa1740b5570 2024-09-15T23:37:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.689+0000 7fa1717fa640 1 -- 172.21.15.7:0/1839966772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa16403c6d0 con 0x7fa1740b5570 2024-09-15T23:37:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.689+0000 7fa1717fa640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa16006c2e0 0x7fa16006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.689+0000 7fa1717fa640 1 -- 172.21.15.7:0/1839966772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa1640abe80 con 0x7fa1740b5570 2024-09-15T23:37:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.689+0000 7fa173fff640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa16006c2e0 0x7fa16006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.690+0000 7fa173fff640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa16006c2e0 0x7fa16006e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1740cc390 tx=0x7fa168007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:26.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.694+0000 7fa1717fa640 1 -- 172.21.15.7:0/1839966772 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa164079e10 con 0x7fa1740b5570 2024-09-15T23:37:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:26.916+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fa1740cf830 con 0x7fa1740b5570 2024-09-15T23:37:27.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.050+0000 7fa1717fa640 1 -- 172.21.15.7:0/1839966772 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) ==== 92+0+0 (secure 0 0 0) 0x7fa16407e110 con 0x7fa1740b5570 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa16006c2e0 msgr2=0x7fa16006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa16006c2e0 0x7fa16006e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1740cc390 tx=0x7fa168007d30 comp rx=0 tx=0).stop 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1740b5570 msgr2=0x7fa17415a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1740b5570 0x7fa17415a780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa1640097a0 tx=0x7fa1640046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 shutdown_connections 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa174161f20 0x7fa17415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7fa16006c2e0 0x7fa16006e7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1740b5c40 0x7fa17414dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 --2- 172.21.15.7:0/1839966772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1740b5570 0x7fa17415a780 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa1640097a0 tx=0x7fa1640046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.053+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 >> 172.21.15.7:0/1839966772 conn(0x7fa17408bfd0 msgr2=0x7fa1740b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.055+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 shutdown_connections 2024-09-15T23:37:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.055+0000 7fa17af86640 1 -- 172.21.15.7:0/1839966772 wait complete. 2024-09-15T23:37:27.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-15T23:37:27.171 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:27.171+0000 7f4723b26040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:37:27.211 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:27.210+0000 7fbd3d00e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:37:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.212+0000 7f8dd51fd640 1 Processor -- start 2024-09-15T23:37:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.212+0000 7f8dd51fd640 1 -- start start 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.213+0000 7f8dd51fd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd015fc60 0x7f8dd00d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.213+0000 7f8dd51fd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8dd00d3040 0x7f8dd00b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.213+0000 7f8dd51fd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd0160040 con 0x7f8dd00b20e0 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.213+0000 7f8dd51fd640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd00cd6a0 con 0x7f8dd015fc60 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.213+0000 7f8dd51fd640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd00d14a0 con 0x7f8dd00d3040 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.213+0000 7f8dced9d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd015fc60 0x7f8dd00d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.213+0000 7f8dcf59e640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8dd00b20e0 0x7f8dd00cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39856/0 (socket says 172.21.15.7:39856) 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.213+0000 7f8dcf59e640 1 -- 172.21.15.7:0/3199782353 learned_addr learned my addr 172.21.15.7:0/3199782353 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.214+0000 7f8dce59c640 1 --2- 172.21.15.7:0/3199782353 >> v2:172.21.15.18:3300/0 conn(0x7f8dd00d3040 0x7f8dd00b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.214+0000 7f8dced9d640 1 -- 172.21.15.7:0/3199782353 >> v2:172.21.15.18:3300/0 conn(0x7f8dd00d3040 msgr2=0x7f8dd00b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.214+0000 7f8dced9d640 1 --2- 172.21.15.7:0/3199782353 >> v2:172.21.15.18:3300/0 conn(0x7f8dd00d3040 0x7f8dd00b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.214+0000 7f8dced9d640 1 -- 172.21.15.7:0/3199782353 >> v1:172.21.15.7:6789/0 conn(0x7f8dd00b20e0 legacy=0x7f8dd00cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.214+0000 7f8dcdd9b640 1 -- 172.21.15.7:0/3199782353 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3513990132 0 0) 0x7f8dd0160040 con 0x7f8dd00b20e0 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.214+0000 7f8dced9d640 1 -- 172.21.15.7:0/3199782353 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd009b7d0 con 0x7f8dd015fc60 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dced9d640 1 --2- 172.21.15.7:0/3199782353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd015fc60 0x7f8dd00d0f60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00cee90 tx=0x7f8dc002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc1cd1391f79671 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dcdd9b640 1 -- 172.21.15.7:0/3199782353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0037070 con 0x7f8dd015fc60 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dcdd9b640 1 -- 172.21.15.7:0/3199782353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc0032440 con 0x7f8dd015fc60 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dcdd9b640 1 -- 172.21.15.7:0/3199782353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0032810 con 0x7f8dd015fc60 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dd51fd640 1 -- 172.21.15.7:0/3199782353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd015fc60 msgr2=0x7f8dd00d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dd51fd640 1 --2- 172.21.15.7:0/3199782353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd015fc60 0x7f8dd00d0f60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00cee90 tx=0x7f8dc002a1a0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dd51fd640 1 -- 172.21.15.7:0/3199782353 shutdown_connections 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dd51fd640 1 --2- 172.21.15.7:0/3199782353 >> v2:172.21.15.18:3300/0 conn(0x7f8dd00d3040 0x7f8dd00b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dd51fd640 1 --2- 172.21.15.7:0/3199782353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd015fc60 0x7f8dd00d0f60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dd51fd640 1 -- 172.21.15.7:0/3199782353 >> 172.21.15.7:0/3199782353 conn(0x7f8dd00d2440 msgr2=0x7f8dd00d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dd51fd640 1 -- 172.21.15.7:0/3199782353 shutdown_connections 2024-09-15T23:37:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.215+0000 7f8dd51fd640 1 -- 172.21.15.7:0/3199782353 wait complete. 2024-09-15T23:37:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.217+0000 7f8dd51fd640 1 Processor -- start 2024-09-15T23:37:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.217+0000 7f8dd51fd640 1 -- start start 2024-09-15T23:37:27.220 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:27.213+0000 7f4723b26040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:37:27.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dd51fd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd00b20e0 0x7f8dd0153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dd51fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd00d3040 0x7f8dd00b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dd51fd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd015fc60 0x7f8dd00afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dd51fd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd00ccfe0 con 0x7f8dd00d3040 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dd51fd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd00cd330 con 0x7f8dd00b20e0 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dd51fd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd00ce240 con 0x7f8dd015fc60 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dce59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd00d3040 0x7f8dd00b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dce59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd00d3040 0x7f8dd00b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53550/0 (socket says 172.21.15.7:53550) 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dce59c640 1 -- 172.21.15.7:0/459716201 learned_addr learned my addr 172.21.15.7:0/459716201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dced9d640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd00b20e0 0x7f8dd0153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dcf59e640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd015fc60 0x7f8dd00afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dce59c640 1 -- 172.21.15.7:0/459716201 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd015fc60 msgr2=0x7f8dd00afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dce59c640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd015fc60 0x7f8dd00afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dce59c640 1 -- 172.21.15.7:0/459716201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd00b20e0 msgr2=0x7f8dd0153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dce59c640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd00b20e0 0x7f8dd0153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dce59c640 1 -- 172.21.15.7:0/459716201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd0153540 con 0x7f8dd00d3040 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dced9d640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd00b20e0 0x7f8dd0153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8dce59c640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd00d3040 0x7f8dd00b7a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400e9e0 tx=0x7f8dc400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.218+0000 7f8db77fe640 1 -- 172.21.15.7:0/459716201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4017070 con 0x7f8dd00d3040 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.219+0000 7f8db77fe640 1 -- 172.21.15.7:0/459716201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc400c9d0 con 0x7f8dd00d3040 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.219+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd00b62f0 con 0x7f8dd00d3040 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.219+0000 7f8db77fe640 1 -- 172.21.15.7:0/459716201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc400cdc0 con 0x7f8dd00d3040 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.219+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd00b6f40 con 0x7f8dd00d3040 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.220+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd00b1ba0 con 0x7f8dd00d3040 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.220+0000 7f8db77fe640 1 -- 172.21.15.7:0/459716201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 6) ==== 90863+0+0 (secure 0 0 0) 0x7f8dc401a070 con 0x7f8dd00d3040 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.221+0000 7f8db77fe640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dbc06c300 0x7f8dbc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.221+0000 7f8dced9d640 1 -- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dbc06c300 msgr2=0x7f8dbc06e7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/3993537392 2024-09-15T23:37:27.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.221+0000 7f8db77fe640 1 -- 172.21.15.7:0/459716201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f8dc408b560 con 0x7f8dd00d3040 2024-09-15T23:37:27.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.221+0000 7f8dced9d640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dbc06c300 0x7f8dbc06e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:27.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.225+0000 7f8db77fe640 1 -- 172.21.15.7:0/459716201 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8dc40594f0 con 0x7f8dd00d3040 2024-09-15T23:37:27.257 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:27.256+0000 7fbd3d00e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:37:27.351 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:27.350+0000 7f4723b26040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:37:27.397 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:27.396+0000 7fbd3d00e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:37:27.401 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:27.401+0000 7f4723b26040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:37:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.401+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f8dd00cf130 con 0x7f8dd00d3040 2024-09-15T23:37:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.404+0000 7f8db77fe640 1 -- 172.21.15.7:0/459716201 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) ==== 138+0+0 (secure 0 0 0) 0x7f8dc405d7f0 con 0x7f8dd00d3040 2024-09-15T23:37:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dbc06c300 msgr2=0x7f8dbc06e7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dbc06c300 0x7f8dbc06e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd00d3040 msgr2=0x7f8dd00b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd00d3040 0x7f8dd00b7a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400e9e0 tx=0x7f8dc400eea0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 shutdown_connections 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f8dbc06c300 0x7f8dbc06e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8dd015fc60 0x7f8dd00afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd00d3040 0x7f8dd00b7a20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 --2- 172.21.15.7:0/459716201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8dd00b20e0 0x7f8dd0153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.406+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 >> 172.21.15.7:0/459716201 conn(0x7f8dd00d2440 msgr2=0x7f8dd00b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.407+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 shutdown_connections 2024-09-15T23:37:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.407+0000 7f8dd51fd640 1 -- 172.21.15.7:0/459716201 wait complete. 2024-09-15T23:37:27.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-15T23:37:27.449 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:27.449+0000 7fbd3d00e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:37:27.515 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:27.514+0000 7f4723b26040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:37:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.527+0000 7f0635949640 1 Processor -- start 2024-09-15T23:37:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.527+0000 7f0635949640 1 -- start start 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.527+0000 7f0635949640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 0x7f0630162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.527+0000 7f0635949640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f06300d24e0 0x7f0630162890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.527+0000 7f0635949640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06300d2980 con 0x7f063015e990 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.527+0000 7f0635949640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f06300d2c40 con 0x7f06300d24e0 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.527+0000 7f0635949640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f06300d2f20 con 0x7f063015e5b0 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062f7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f063015e990 0x7f06300d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39860/0 (socket says 172.21.15.7:39860) 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062f7fe640 1 -- 172.21.15.7:0/821096907 learned_addr learned my addr 172.21.15.7:0/821096907 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062e7fc640 1 --2- 172.21.15.7:0/821096907 >> v2:172.21.15.18:3300/0 conn(0x7f06300d24e0 0x7f0630162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062effd640 1 --2- 172.21.15.7:0/821096907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 0x7f0630162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062dffb640 1 -- 172.21.15.7:0/821096907 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2421172456 0 0) 0x7f06300d2980 con 0x7f063015e990 2024-09-15T23:37:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062dffb640 1 -- 172.21.15.7:0/821096907 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0618003520 con 0x7f063015e990 2024-09-15T23:37:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062e7fc640 1 -- 172.21.15.7:0/821096907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 msgr2=0x7f0630162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062e7fc640 1 --2- 172.21.15.7:0/821096907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 0x7f0630162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062e7fc640 1 -- 172.21.15.7:0/821096907 >> v1:172.21.15.7:6789/0 conn(0x7f063015e990 legacy=0x7f06300d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.528+0000 7f062e7fc640 1 -- 172.21.15.7:0/821096907 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06300d3200 con 0x7f06300d24e0 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.529+0000 7f062e7fc640 1 --2- 172.21.15.7:0/821096907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06300d24e0 0x7f0630162890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0624009790 tx=0x7f062402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a731b01ec19278b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.529+0000 7f062dffb640 1 -- 172.21.15.7:0/821096907 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624037070 con 0x7f06300d24e0 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.529+0000 7f062dffb640 1 -- 172.21.15.7:0/821096907 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0624032440 con 0x7f06300d24e0 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.529+0000 7f062dffb640 1 -- 172.21.15.7:0/821096907 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624032810 con 0x7f06300d24e0 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.530+0000 7f0635949640 1 -- 172.21.15.7:0/821096907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06300d24e0 msgr2=0x7f0630162890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.530+0000 7f0635949640 1 --2- 172.21.15.7:0/821096907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06300d24e0 0x7f0630162890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0624009790 tx=0x7f062402a180 comp rx=0 tx=0).stop 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.530+0000 7f0635949640 1 -- 172.21.15.7:0/821096907 shutdown_connections 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.530+0000 7f0635949640 1 --2- 172.21.15.7:0/821096907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06300d24e0 0x7f0630162890 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.530+0000 7f0635949640 1 --2- 172.21.15.7:0/821096907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 0x7f0630162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.530+0000 7f0635949640 1 -- 172.21.15.7:0/821096907 >> 172.21.15.7:0/821096907 conn(0x7f063008bfd0 msgr2=0x7f06300d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.530+0000 7f0635949640 1 -- 172.21.15.7:0/821096907 shutdown_connections 2024-09-15T23:37:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.530+0000 7f0635949640 1 -- 172.21.15.7:0/821096907 wait complete. 2024-09-15T23:37:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.531+0000 7f0635949640 1 Processor -- start 2024-09-15T23:37:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.531+0000 7f0635949640 1 -- start start 2024-09-15T23:37:27.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f0635949640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06300d24e0 0x7f063006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f0635949640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 0x7f0630151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f0635949640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f063015e990 0x7f06301516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f0635949640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06300b3b40 con 0x7f063015e990 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f0635949640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f06300b39c0 con 0x7f06300d24e0 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f0635949640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f06300d0430 con 0x7f063015e5b0 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f063015e990 0x7f06301516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f063015e990 0x7f06301516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53556/0 (socket says 172.21.15.7:53556) 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062e7fc640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 0x7f0630151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062f7fe640 1 -- 172.21.15.7:0/2457174515 learned_addr learned my addr 172.21.15.7:0/2457174515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062effd640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06300d24e0 0x7f063006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062f7fe640 1 -- 172.21.15.7:0/2457174515 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 msgr2=0x7f0630151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062f7fe640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 0x7f0630151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062f7fe640 1 -- 172.21.15.7:0/2457174515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06300d24e0 msgr2=0x7f063006da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062f7fe640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06300d24e0 0x7f063006da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.532+0000 7f062f7fe640 1 -- 172.21.15.7:0/2457174515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06300cc150 con 0x7f063015e990 2024-09-15T23:37:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.533+0000 7f062f7fe640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f063015e990 0x7f06301516a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f062800ca70 tx=0x7f062800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.533+0000 7f0617fff640 1 -- 172.21.15.7:0/2457174515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628017070 con 0x7f063015e990 2024-09-15T23:37:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.533+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06300ce2f0 con 0x7f063015e990 2024-09-15T23:37:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.533+0000 7f0617fff640 1 -- 172.21.15.7:0/2457174515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062800ab70 con 0x7f063015e990 2024-09-15T23:37:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.533+0000 7f0617fff640 1 -- 172.21.15.7:0/2457174515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628012400 con 0x7f063015e990 2024-09-15T23:37:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.533+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06300b7820 con 0x7f063015e990 2024-09-15T23:37:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.534+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0630162350 con 0x7f063015e990 2024-09-15T23:37:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.534+0000 7f0617fff640 1 -- 172.21.15.7:0/2457174515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 6) ==== 90863+0+0 (secure 0 0 0) 0x7f062801a070 con 0x7f063015e990 2024-09-15T23:37:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.535+0000 7f0617fff640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f061c06c300 0x7f061c06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.535+0000 7f062effd640 1 -- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f061c06c300 msgr2=0x7f061c06e7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/3993537392 2024-09-15T23:37:27.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.535+0000 7f0617fff640 1 -- 172.21.15.7:0/2457174515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f062808b1d0 con 0x7f063015e990 2024-09-15T23:37:27.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.535+0000 7f062effd640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f061c06c300 0x7f061c06e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:27.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.539+0000 7f0617fff640 1 -- 172.21.15.7:0/2457174515 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0628059170 con 0x7f063015e990 2024-09-15T23:37:27.567 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:27.566+0000 7fbd3d00e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:37:27.619 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:27.612+0000 7f4723b26040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:37:27.667 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:27.666+0000 7fbd3d00e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:37:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.717+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f06300ba560 con 0x7f063015e990 2024-09-15T23:37:27.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.720+0000 7f0617fff640 1 -- 172.21.15.7:0/2457174515 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) ==== 133+0+0 (secure 0 0 0) 0x7f062805d470 con 0x7f063015e990 2024-09-15T23:37:27.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f061c06c300 msgr2=0x7f061c06e7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:27.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f061c06c300 0x7f061c06e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f063015e990 msgr2=0x7f06301516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f063015e990 0x7f06301516a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f062800ca70 tx=0x7f062800cf30 comp rx=0 tx=0).stop 2024-09-15T23:37:27.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 shutdown_connections 2024-09-15T23:37:27.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f063015e990 0x7f06301516a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7f061c06c300 0x7f061c06e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f063015e5b0 0x7f0630151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 --2- 172.21.15.7:0/2457174515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06300d24e0 0x7f063006da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.722+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 >> 172.21.15.7:0/2457174515 conn(0x7f063008bfd0 msgr2=0x7f06300d4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:27.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.723+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 shutdown_connections 2024-09-15T23:37:27.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.723+0000 7f0635949640 1 -- 172.21.15.7:0/2457174515 wait complete. 2024-09-15T23:37:27.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:27.735 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-09-15T23:37:27.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2024-09-15T23:37:27.893 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:27.892+0000 7f4723b26040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:37:27.939 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:27.937+0000 7f4723b26040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:37:27.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.942+0000 7ff40e227640 1 Processor -- start 2024-09-15T23:37:27.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.943+0000 7ff40e227640 1 -- start start 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.943+0000 7ff40e227640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff4080d24e0 0x7ff408162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.943+0000 7ff40e227640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff40815e990 0x7ff4080cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.943+0000 7ff40e227640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4080d2980 con 0x7ff40815e5b0 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.943+0000 7ff40e227640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff4080d2c40 con 0x7ff4080d24e0 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.943+0000 7ff40e227640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff4080d2f20 con 0x7ff40815e990 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff406ffd640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff4080d24e0 0x7ff408162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff4077fe640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7ff40815e5b0 0x7ff408162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:40648/0 (socket says 172.21.15.7:40648) 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff4077fe640 1 -- 172.21.15.7:0/159074775 learned_addr learned my addr 172.21.15.7:0/159074775 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff407fff640 1 --2- 172.21.15.7:0/159074775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff40815e990 0x7ff4080cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff4067fc640 1 -- 172.21.15.7:0/159074775 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2758052838 0 0) 0x7ff4080d2980 con 0x7ff40815e5b0 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff4067fc640 1 -- 172.21.15.7:0/159074775 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3f0003520 con 0x7ff40815e5b0 2024-09-15T23:37:27.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff406ffd640 1 -- 172.21.15.7:0/159074775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff40815e990 msgr2=0x7ff4080cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff406ffd640 1 --2- 172.21.15.7:0/159074775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff40815e990 0x7ff4080cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff406ffd640 1 -- 172.21.15.7:0/159074775 >> v1:172.21.15.18:6789/0 conn(0x7ff40815e5b0 legacy=0x7ff408162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.944+0000 7ff406ffd640 1 -- 172.21.15.7:0/159074775 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4080d3200 con 0x7ff4080d24e0 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.945+0000 7ff406ffd640 1 --2- 172.21.15.7:0/159074775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4080d24e0 0x7ff408162a60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0097c0 tx=0x7ff3fc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c01671c902d6cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.945+0000 7ff4067fc640 1 -- 172.21.15.7:0/159074775 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc037070 con 0x7ff4080d24e0 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.945+0000 7ff4067fc640 1 -- 172.21.15.7:0/159074775 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3fc032440 con 0x7ff4080d24e0 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.945+0000 7ff4067fc640 1 -- 172.21.15.7:0/159074775 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc032810 con 0x7ff4080d24e0 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.946+0000 7ff40e227640 1 -- 172.21.15.7:0/159074775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4080d24e0 msgr2=0x7ff408162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.946+0000 7ff40e227640 1 --2- 172.21.15.7:0/159074775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4080d24e0 0x7ff408162a60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0097c0 tx=0x7ff3fc02a1a0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.946+0000 7ff40e227640 1 -- 172.21.15.7:0/159074775 shutdown_connections 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.946+0000 7ff40e227640 1 --2- 172.21.15.7:0/159074775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff40815e990 0x7ff4080cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.946+0000 7ff40e227640 1 --2- 172.21.15.7:0/159074775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4080d24e0 0x7ff408162a60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.946+0000 7ff40e227640 1 -- 172.21.15.7:0/159074775 >> 172.21.15.7:0/159074775 conn(0x7ff40808bfd0 msgr2=0x7ff4080d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:27.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.946+0000 7ff40e227640 1 -- 172.21.15.7:0/159074775 shutdown_connections 2024-09-15T23:37:27.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.946+0000 7ff40e227640 1 -- 172.21.15.7:0/159074775 wait complete. 2024-09-15T23:37:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.947+0000 7ff40e227640 1 Processor -- start 2024-09-15T23:37:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.947+0000 7ff40e227640 1 -- start start 2024-09-15T23:37:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff40e227640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4080d24e0 0x7ff40814daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff40e227640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff40815e5b0 0x7ff4080ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff40e227640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff40815e990 0x7ff4080b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff40e227640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4080b3b40 con 0x7ff4080d24e0 2024-09-15T23:37:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff40e227640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff4080b39c0 con 0x7ff40815e5b0 2024-09-15T23:37:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff40e227640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff408162fa0 con 0x7ff40815e990 2024-09-15T23:37:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff4077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4080d24e0 0x7ff40814daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff406ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff40815e5b0 0x7ff4080ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff407fff640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff40815e990 0x7ff4080b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff4077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4080d24e0 0x7ff40814daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53564/0 (socket says 172.21.15.7:53564) 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff4077fe640 1 -- 172.21.15.7:0/227132419 learned_addr learned my addr 172.21.15.7:0/227132419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff406ffd640 1 -- 172.21.15.7:0/227132419 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff40815e990 msgr2=0x7ff4080b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff406ffd640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff40815e990 0x7ff4080b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff406ffd640 1 -- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4080d24e0 msgr2=0x7ff40814daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff406ffd640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4080d24e0 0x7ff40814daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff406ffd640 1 -- 172.21.15.7:0/227132419 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4080b4f70 con 0x7ff40815e5b0 2024-09-15T23:37:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.948+0000 7ff4077fe640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4080d24e0 0x7ff40814daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.949+0000 7ff406ffd640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff40815e5b0 0x7ff4080ca7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc031000 tx=0x7ff3fc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.949+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc037040 con 0x7ff40815e5b0 2024-09-15T23:37:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.949+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3fc005a50 con 0x7ff40815e5b0 2024-09-15T23:37:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.949+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4080b5250 con 0x7ff40815e5b0 2024-09-15T23:37:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.949+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc005e00 con 0x7ff40815e5b0 2024-09-15T23:37:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.949+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4080b5810 con 0x7ff40815e5b0 2024-09-15T23:37:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.950+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff408162350 con 0x7ff40815e5b0 2024-09-15T23:37:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.950+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 6) ==== 90863+0+0 (secure 0 0 0) 0x7ff3fc044020 con 0x7ff40815e5b0 2024-09-15T23:37:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.951+0000 7ff3effff640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 0x7ff3f406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.951+0000 7ff4077fe640 1 -- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 msgr2=0x7ff3f406e7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/3993537392 2024-09-15T23:37:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.951+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff3fc0ab6a0 con 0x7ff40815e5b0 2024-09-15T23:37:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.951+0000 7ff4077fe640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 0x7ff3f406e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:27.953 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:27.951+0000 7fbd3d00e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:37:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:27.956+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff3fc079620 con 0x7ff40815e5b0 2024-09-15T23:37:27.999 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:27.999+0000 7fbd3d00e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:37:28.061 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:28.060+0000 7f4723b26040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:37:28.124 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:28.123+0000 7fbd3d00e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:37:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:28.152+0000 7ff4077fe640 1 -- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 msgr2=0x7ff3f406e7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/3993537392 2024-09-15T23:37:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:28.152+0000 7ff4077fe640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 0x7ff3f406e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:37:28.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:28.153+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 --> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff40814dfe0 con 0x7ff3f406c330 2024-09-15T23:37:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:28.553+0000 7ff4077fe640 1 -- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 msgr2=0x7ff3f406e7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/3993537392 2024-09-15T23:37:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:28.553+0000 7ff4077fe640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 0x7ff3f406e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T23:37:28.795 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:28.794+0000 7f4723b26040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:37:28.841 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:28.840+0000 7f4723b26040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:37:28.861 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:28.860+0000 7fbd3d00e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:37:28.907 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:28.906+0000 7fbd3d00e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:37:29.169 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.168+0000 7f4723b26040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:37:29.236 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.235+0000 7fbd3d00e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:37:29.255 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.254+0000 7f4723b26040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:37:29.324 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.323+0000 7fbd3d00e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:37:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:29.354+0000 7ff4077fe640 1 -- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 msgr2=0x7ff3f406e7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/3993537392 2024-09-15T23:37:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:29.354+0000 7ff4077fe640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 0x7ff3f406e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T23:37:29.367 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.365+0000 7f4723b26040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:37:29.411 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.410+0000 7f4723b26040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:37:29.437 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.436+0000 7fbd3d00e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:37:29.481 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.480+0000 7fbd3d00e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:37:29.526 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.525+0000 7f4723b26040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:37:29.580 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.579+0000 7f4723b26040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:37:29.597 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.597+0000 7fbd3d00e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:37:29.623 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.622+0000 7f4723b26040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:37:29.651 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.651+0000 7fbd3d00e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:37:29.695 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.694+0000 7fbd3d00e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:37:29.740 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.739+0000 7f4723b26040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:37:29.793 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.789+0000 7f4723b26040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:37:29.813 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.812+0000 7fbd3d00e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:37:29.840 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.839+0000 7f4723b26040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:37:29.863 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.862+0000 7fbd3d00e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:37:29.894 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.893+0000 7f4723b26040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:37:29.913 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.913+0000 7fbd3d00e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:37:29.938 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.937+0000 7f4723b26040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:37:29.968 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:29.968+0000 7fbd3d00e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:37:29.981 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:29.980+0000 7f4723b26040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:37:30.013 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.012+0000 7fbd3d00e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:37:30.057 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.056+0000 7fbd3d00e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:37:30.572 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:30.571+0000 7f4723b26040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:37:30.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:30.630+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 7) ==== 90863+0+0 (secure 0 0 0) 0x7ff3fc075c90 con 0x7ff40815e5b0 2024-09-15T23:37:30.657 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.656+0000 7fbd3d00e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:37:30.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:30.665+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mon.1 v2:172.21.15.18:3300/0 8 ==== mgrmap(e 8) ==== 90574+0+0 (secure 0 0 0) 0x7ff3fc0959a0 con 0x7ff40815e5b0 2024-09-15T23:37:30.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:30.665+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 msgr2=0x7ff3f406e7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:30.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:30.665+0000 7ff3effff640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/3993537392,v1:172.21.15.7:6833/3993537392] conn(0x7ff3f406c330 0x7ff3f406e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:30.790 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcbb3f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:30.790 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcbb3f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:30.790 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcbb3f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:30.790 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcbb3f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:30.790 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcbb3f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:30.790 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcb73e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:30.791 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcb73e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:30.791 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcb73e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:30.791 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcb73e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:30.791 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:30.789+0000 7fbcb73e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:31.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.670+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mon.1 v2:172.21.15.18:3300/0 9 ==== mgrmap(e 9) ==== 90701+0+0 (secure 0 0 0) 0x7ff3fc076560 con 0x7ff40815e5b0 2024-09-15T23:37:31.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.671+0000 7ff3effff640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7ff3f4074830 0x7ff3f4076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:31.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.671+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 --> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff3fc081aa0 con 0x7ff3f4074830 2024-09-15T23:37:31.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.672+0000 7ff4077fe640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7ff3f4074830 0x7ff3f4076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:31.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.675+0000 7ff4077fe640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7ff3f4074830 0x7ff3f4076c20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8005880 tx=0x7ff3f8007d90 comp rx=0 tx=0).ready entity=mgr.5067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.749+0000 7ff3effff640 1 -- 172.21.15.7:0/227132419 <== mgr.5067 v2:172.21.15.7:6832/4033779377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff3fc081aa0 con 0x7ff3f4074830 2024-09-15T23:37:31.750 INFO:teuthology.orchestra.run.smithi007.stdout:Schedule set for path snap_test_dir1 2024-09-15T23:37:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.751+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7ff3f4074830 msgr2=0x7ff3f4076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.751+0000 7ff40e227640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7ff3f4074830 0x7ff3f4076c20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8005880 tx=0x7ff3f8007d90 comp rx=0 tx=0).stop 2024-09-15T23:37:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.751+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff40815e5b0 msgr2=0x7ff4080ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.751+0000 7ff40e227640 1 --2- 172.21.15.7:0/227132419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff40815e5b0 0x7ff4080ca7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc031000 tx=0x7ff3fc0046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.751+0000 7ff407fff640 1 -- 172.21.15.7:0/227132419 reap_dead start 2024-09-15T23:37:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.751+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 shutdown_connections 2024-09-15T23:37:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.751+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 >> 172.21.15.7:0/227132419 conn(0x7ff40808bfd0 msgr2=0x7ff408159380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.751+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 shutdown_connections 2024-09-15T23:37:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.751+0000 7ff40e227640 1 -- 172.21.15.7:0/227132419 wait complete. 2024-09-15T23:37:31.763 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-09-15T23:37:31.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2024-09-15T23:37:31.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.872+0000 7fddd22af640 1 Processor -- start 2024-09-15T23:37:31.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.872+0000 7fddd22af640 1 -- start start 2024-09-15T23:37:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd22af640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fddcc0d24e0 0x7fddcc162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd22af640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fddcc15e990 0x7fddcc0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd22af640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddcc0d2980 con 0x7fddcc15e5b0 2024-09-15T23:37:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd22af640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fddcc0d2c40 con 0x7fddcc15e990 2024-09-15T23:37:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd22af640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fddcc0d2f20 con 0x7fddcc0d24e0 2024-09-15T23:37:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd084c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fddcc15e990 0x7fddcc0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddcbfff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fddcc15e5b0 0x7fddcc162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39868/0 (socket says 172.21.15.7:39868) 2024-09-15T23:37:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddcbfff640 1 -- 172.21.15.7:0/3631054153 learned_addr learned my addr 172.21.15.7:0/3631054153 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddcb7fe640 1 --2- 172.21.15.7:0/3631054153 >> v2:172.21.15.18:3300/0 conn(0x7fddcc0d24e0 0x7fddcc162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd084c640 1 -- 172.21.15.7:0/3631054153 >> v2:172.21.15.18:3300/0 conn(0x7fddcc0d24e0 msgr2=0x7fddcc162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd084c640 1 --2- 172.21.15.7:0/3631054153 >> v2:172.21.15.18:3300/0 conn(0x7fddcc0d24e0 0x7fddcc162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd084c640 1 -- 172.21.15.7:0/3631054153 >> v1:172.21.15.7:6789/0 conn(0x7fddcc15e5b0 legacy=0x7fddcc162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddcaffd640 1 -- 172.21.15.7:0/3631054153 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4005817161 0 0) 0x7fddcc0d2980 con 0x7fddcc15e5b0 2024-09-15T23:37:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddd084c640 1 -- 172.21.15.7:0/3631054153 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc0d3200 con 0x7fddcc15e990 2024-09-15T23:37:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.873+0000 7fddcb7fe640 1 --2- 172.21.15.7:0/3631054153 >> v2:172.21.15.18:3300/0 conn(0x7fddcc0d24e0 0x7fddcc162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.874+0000 7fddd084c640 1 --2- 172.21.15.7:0/3631054153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e990 0x7fddcc0cf420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009d90 tx=0x7fddc402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a368c6b7dc3dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.874+0000 7fddcaffd640 1 -- 172.21.15.7:0/3631054153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4007740 con 0x7fddcc15e990 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.874+0000 7fddcaffd640 1 -- 172.21.15.7:0/3631054153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc40078e0 con 0x7fddcc15e990 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.874+0000 7fddcaffd640 1 -- 172.21.15.7:0/3631054153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4007cb0 con 0x7fddcc15e990 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.874+0000 7fddd22af640 1 -- 172.21.15.7:0/3631054153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e990 msgr2=0x7fddcc0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.874+0000 7fddd22af640 1 --2- 172.21.15.7:0/3631054153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e990 0x7fddcc0cf420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009d90 tx=0x7fddc402a6a0 comp rx=0 tx=0).stop 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.875+0000 7fddd22af640 1 -- 172.21.15.7:0/3631054153 shutdown_connections 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.875+0000 7fddd22af640 1 --2- 172.21.15.7:0/3631054153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e990 0x7fddcc0cf420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.875+0000 7fddd22af640 1 --2- 172.21.15.7:0/3631054153 >> v2:172.21.15.18:3300/0 conn(0x7fddcc0d24e0 0x7fddcc162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.875+0000 7fddd22af640 1 -- 172.21.15.7:0/3631054153 >> 172.21.15.7:0/3631054153 conn(0x7fddcc08bfd0 msgr2=0x7fddcc0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.875+0000 7fddd22af640 1 -- 172.21.15.7:0/3631054153 shutdown_connections 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.875+0000 7fddd22af640 1 -- 172.21.15.7:0/3631054153 wait complete. 2024-09-15T23:37:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.876+0000 7fddd22af640 1 Processor -- start 2024-09-15T23:37:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.876+0000 7fddd22af640 1 -- start start 2024-09-15T23:37:31.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddd22af640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fddcc0d24e0 0x7fddcc1f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:31.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddd22af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e5b0 0x7fddcc0b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddd22af640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc15e990 0x7fddcc1f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddd22af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc0b3b40 con 0x7fddcc15e5b0 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddd22af640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc0b39c0 con 0x7fddcc15e990 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddd22af640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fddcc162fa0 con 0x7fddcc0d24e0 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e5b0 0x7fddcc0b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e5b0 0x7fddcc0b87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53712/0 (socket says 172.21.15.7:53712) 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcb7fe640 1 -- 172.21.15.7:0/554959949 learned_addr learned my addr 172.21.15.7:0/554959949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddd084c640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc15e990 0x7fddcc1f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcbfff640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fddcc0d24e0 0x7fddcc1f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcb7fe640 1 -- 172.21.15.7:0/554959949 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fddcc0d24e0 msgr2=0x7fddcc1f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcb7fe640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fddcc0d24e0 0x7fddcc1f74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcb7fe640 1 -- 172.21.15.7:0/554959949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc15e990 msgr2=0x7fddcc1f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcb7fe640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc15e990 0x7fddcc1f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcb7fe640 1 -- 172.21.15.7:0/554959949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc1f5450 con 0x7fddcc15e5b0 2024-09-15T23:37:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.877+0000 7fddcb7fe640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e5b0 0x7fddcc0b87f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e810 tx=0x7fddc000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.878+0000 7fddc8ff9640 1 -- 172.21.15.7:0/554959949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc000f040 con 0x7fddcc15e5b0 2024-09-15T23:37:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.878+0000 7fddc8ff9640 1 -- 172.21.15.7:0/554959949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc0004230 con 0x7fddcc15e5b0 2024-09-15T23:37:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.878+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddcc0b8e90 con 0x7fddcc15e5b0 2024-09-15T23:37:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.878+0000 7fddc8ff9640 1 -- 172.21.15.7:0/554959949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0005990 con 0x7fddcc15e5b0 2024-09-15T23:37:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.878+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddcc0b9450 con 0x7fddcc15e5b0 2024-09-15T23:37:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.879+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddcc068f20 con 0x7fddcc15e5b0 2024-09-15T23:37:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.879+0000 7fddc8ff9640 1 -- 172.21.15.7:0/554959949 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 90701+0+0 (secure 0 0 0) 0x7fddc0005b30 con 0x7fddcc15e5b0 2024-09-15T23:37:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.880+0000 7fddc8ff9640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fddb806c150 0x7fddb806e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.880+0000 7fddcbfff640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fddb806c150 0x7fddb806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.880+0000 7fddc8ff9640 1 -- 172.21.15.7:0/554959949 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fddc008b360 con 0x7fddcc15e5b0 2024-09-15T23:37:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.880+0000 7fddcbfff640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fddb806c150 0x7fddb806e610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fddbc000f60 tx=0x7fddbc007d50 comp rx=0 tx=0).ready entity=mgr.5067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:31.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:31.885+0000 7fddc8ff9640 1 -- 172.21.15.7:0/554959949 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fddc0020020 con 0x7fddcc15e5b0 2024-09-15T23:37:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.087+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 --> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) -- 0x7fddcc0ba560 con 0x7fddb806c150 2024-09-15T23:37:32.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.138+0000 7fddc8ff9640 1 -- 172.21.15.7:0/554959949 <== mgr.5067 v2:172.21.15.7:6832/4033779377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+39 (secure 0 0 0) 0x7fddcc0ba560 con 0x7fddb806c150 2024-09-15T23:37:32.139 INFO:teuthology.orchestra.run.smithi007.stdout:Retention added to path snap_test_dir1 2024-09-15T23:37:32.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fddb806c150 msgr2=0x7fddb806e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fddb806c150 0x7fddb806e610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fddbc000f60 tx=0x7fddbc007d50 comp rx=0 tx=0).stop 2024-09-15T23:37:32.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e5b0 msgr2=0x7fddcc0b87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e5b0 0x7fddcc0b87f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e810 tx=0x7fddc000ecd0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 shutdown_connections 2024-09-15T23:37:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fddb806c150 0x7fddb806e610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc15e990 0x7fddcc1f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc15e5b0 0x7fddcc0b87f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 --2- 172.21.15.7:0/554959949 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fddcc0d24e0 0x7fddcc1f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 >> 172.21.15.7:0/554959949 conn(0x7fddcc08bfd0 msgr2=0x7fddcc159560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 shutdown_connections 2024-09-15T23:37:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.140+0000 7fddd22af640 1 -- 172.21.15.7:0/554959949 wait complete. 2024-09-15T23:37:32.152 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2024-09-15T23:37:32.152 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.customsnapkernel/scheduled-2024-09-15-23_38_00 in ~28s... 2024-09-15T23:37:32.152 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2024-09-15T23:37:32.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2024-09-15T23:37:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.255+0000 7f293a786640 1 Processor -- start 2024-09-15T23:37:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.255+0000 7f293a786640 1 -- start start 2024-09-15T23:37:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f293a786640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2934160120 0x7f29340d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f293a786640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29340d1120 con 0x7f29340d0c10 2024-09-15T23:37:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f293a786640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29340d1400 con 0x7f293409aa40 2024-09-15T23:37:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f293a786640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2934070300 con 0x7f2934160120 2024-09-15T23:37:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f29337fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2934160120 0x7f29340d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f2938d23640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f29340d0c10 0x7f2934070790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39884/0 (socket says 172.21.15.7:39884) 2024-09-15T23:37:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f2938d23640 1 -- 172.21.15.7:0/3986864975 learned_addr learned my addr 172.21.15.7:0/3986864975 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f29337fe640 1 -- 172.21.15.7:0/3986864975 >> v1:172.21.15.18:6789/0 conn(0x7f293409aa40 legacy=0x7f293409ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f29337fe640 1 -- 172.21.15.7:0/3986864975 >> v1:172.21.15.7:6789/0 conn(0x7f29340d0c10 legacy=0x7f2934070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f2932ffd640 1 -- 172.21.15.7:0/3986864975 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121096152 0 0) 0x7f29340d1120 con 0x7f29340d0c10 2024-09-15T23:37:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.256+0000 7f29337fe640 1 -- 172.21.15.7:0/3986864975 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29340d16e0 con 0x7f2934160120 2024-09-15T23:37:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.257+0000 7f29337fe640 1 --2- 172.21.15.7:0/3986864975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 0x7f29340d06d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f29280097a0 tx=0x7f292802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9f5ad9af8e331b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.257+0000 7f2932ffd640 1 -- 172.21.15.7:0/3986864975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928037070 con 0x7f2934160120 2024-09-15T23:37:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.257+0000 7f2932ffd640 1 -- 172.21.15.7:0/3986864975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2928032440 con 0x7f2934160120 2024-09-15T23:37:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.257+0000 7f2932ffd640 1 -- 172.21.15.7:0/3986864975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928032810 con 0x7f2934160120 2024-09-15T23:37:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.258+0000 7f293a786640 1 -- 172.21.15.7:0/3986864975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 msgr2=0x7f29340d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.258+0000 7f293a786640 1 --2- 172.21.15.7:0/3986864975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 0x7f29340d06d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f29280097a0 tx=0x7f292802a1e0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.258+0000 7f293a786640 1 -- 172.21.15.7:0/3986864975 shutdown_connections 2024-09-15T23:37:32.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.258+0000 7f293a786640 1 --2- 172.21.15.7:0/3986864975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 0x7f29340d06d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.258+0000 7f293a786640 1 -- 172.21.15.7:0/3986864975 >> 172.21.15.7:0/3986864975 conn(0x7f29340d2420 msgr2=0x7f29340d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:32.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.258+0000 7f293a786640 1 -- 172.21.15.7:0/3986864975 shutdown_connections 2024-09-15T23:37:32.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.258+0000 7f293a786640 1 -- 172.21.15.7:0/3986864975 wait complete. 2024-09-15T23:37:32.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.259+0000 7f293a786640 1 Processor -- start 2024-09-15T23:37:32.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.259+0000 7f293a786640 1 -- start start 2024-09-15T23:37:32.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f293a786640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f293409aa40 0x7f293414ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f293a786640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29340d0c10 0x7f29340b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f293a786640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 0x7f29340b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f293a786640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2934161f70 con 0x7f2934160120 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f293a786640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f293415f0a0 con 0x7f29340d0c10 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f293a786640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f29341620f0 con 0x7f293409aa40 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f2938d23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 0x7f29340b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f2938d23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 0x7f29340b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53726/0 (socket says 172.21.15.7:53726) 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f2938d23640 1 -- 172.21.15.7:0/2730514498 learned_addr learned my addr 172.21.15.7:0/2730514498 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f29337fe640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29340d0c10 0x7f29340b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f2933fff640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f293409aa40 0x7f293414ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f2938d23640 1 -- 172.21.15.7:0/2730514498 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f293409aa40 msgr2=0x7f293414ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f2938d23640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f293409aa40 0x7f293414ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f2938d23640 1 -- 172.21.15.7:0/2730514498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29340d0c10 msgr2=0x7f29340b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f2938d23640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29340d0c10 0x7f29340b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.260+0000 7f2938d23640 1 -- 172.21.15.7:0/2730514498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29340b7020 con 0x7f2934160120 2024-09-15T23:37:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.261+0000 7f2938d23640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 0x7f29340b2f20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c730 tx=0x7f292c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.261+0000 7f2930ff9640 1 -- 172.21.15.7:0/2730514498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c018070 con 0x7f2934160120 2024-09-15T23:37:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.261+0000 7f2930ff9640 1 -- 172.21.15.7:0/2730514498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292c00ce40 con 0x7f2934160120 2024-09-15T23:37:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.261+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29340b7300 con 0x7f2934160120 2024-09-15T23:37:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.261+0000 7f2930ff9640 1 -- 172.21.15.7:0/2730514498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c00a860 con 0x7f2934160120 2024-09-15T23:37:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.261+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29340b7ba0 con 0x7f2934160120 2024-09-15T23:37:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.262+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29340b74d0 con 0x7f2934160120 2024-09-15T23:37:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.262+0000 7f2930ff9640 1 -- 172.21.15.7:0/2730514498 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 90701+0+0 (secure 0 0 0) 0x7f292c01b070 con 0x7f2934160120 2024-09-15T23:37:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.263+0000 7f2930ff9640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f292006c150 0x7f292006e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.263+0000 7f2933fff640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f292006c150 0x7f292006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.263+0000 7f2930ff9640 1 -- 172.21.15.7:0/2730514498 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f292c08a980 con 0x7f2934160120 2024-09-15T23:37:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.263+0000 7f2933fff640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f292006c150 0x7f292006e610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2924006fd0 tx=0x7f2924009060 comp rx=0 tx=0).ready entity=mgr.5067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:32.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.267+0000 7f2930ff9640 1 -- 172.21.15.7:0/2730514498 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f292c058a80 con 0x7f2934160120 2024-09-15T23:37:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.462+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 --> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29340d1120 con 0x7f292006c150 2024-09-15T23:37:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.468+0000 7f2930ff9640 1 -- 172.21.15.7:0/2730514498 <== mgr.5067 v2:172.21.15.7:6832/4033779377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+72 (secure 0 0 0) 0x7f29340d1120 con 0x7f292006c150 2024-09-15T23:37:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:32.469 INFO:teuthology.orchestra.run.smithi007.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-09-15T23:37:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.470+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f292006c150 msgr2=0x7f292006e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.470+0000 7f293a786640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f292006c150 0x7f292006e610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2924006fd0 tx=0x7f2924009060 comp rx=0 tx=0).stop 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.470+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 msgr2=0x7f29340b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.470+0000 7f293a786640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 0x7f29340b2f20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c730 tx=0x7f292c00cbf0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.470+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 shutdown_connections 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.470+0000 7f293a786640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2934160120 0x7f29340b2f20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.470+0000 7f293a786640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f292006c150 0x7f292006e610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.470+0000 7f293a786640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29340d0c10 0x7f29340b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.470+0000 7f293a786640 1 --2- 172.21.15.7:0/2730514498 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f293409aa40 0x7f293414ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.471+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 >> 172.21.15.7:0/2730514498 conn(0x7f29340d2420 msgr2=0x7f29340ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.471+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 shutdown_connections 2024-09-15T23:37:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.471+0000 7f293a786640 1 -- 172.21.15.7:0/2730514498 wait complete. 2024-09-15T23:37:32.482 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-09-15T23:37:32.482 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2024-09-15T23:37:32.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2024-09-15T23:37:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.585+0000 7fa65a0af640 1 Processor -- start 2024-09-15T23:37:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.585+0000 7fa65a0af640 1 -- start start 2024-09-15T23:37:32.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa65a0af640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6540d1510 0x7fa6540b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa65a0af640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540b5570 0x7fa6540b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa65a0af640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6540d4f80 con 0x7fa654161f20 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa65a0af640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa6540d5540 con 0x7fa6540d1510 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa65a0af640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa6540d4d30 con 0x7fa6540b5570 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa64affd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6540d1510 0x7fa6540b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa6537fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa654161f20 0x7fa6540d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39890/0 (socket says 172.21.15.7:39890) 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa64affd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6540d1510 0x7fa6540b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53734/0 (socket says 172.21.15.7:53734) 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa6537fe640 1 -- 172.21.15.7:0/2269075724 learned_addr learned my addr 172.21.15.7:0/2269075724 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.586+0000 7fa653fff640 1 --2- 172.21.15.7:0/2269075724 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540b5570 0x7fa6540b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.587+0000 7fa64affd640 1 -- 172.21.15.7:0/2269075724 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540b5570 msgr2=0x7fa6540b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.587+0000 7fa64affd640 1 --2- 172.21.15.7:0/2269075724 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540b5570 0x7fa6540b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.587+0000 7fa64affd640 1 -- 172.21.15.7:0/2269075724 >> v1:172.21.15.7:6789/0 conn(0x7fa654161f20 legacy=0x7fa6540d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.587+0000 7fa652ffd640 1 -- 172.21.15.7:0/2269075724 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 810884781 0 0) 0x7fa6540d4f80 con 0x7fa654161f20 2024-09-15T23:37:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.587+0000 7fa64affd640 1 -- 172.21.15.7:0/2269075724 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6540d5260 con 0x7fa6540d1510 2024-09-15T23:37:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.587+0000 7fa64affd640 1 --2- 172.21.15.7:0/2269075724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540d1510 0x7fa6540b3c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0097a0 tx=0x7fa63c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cacb57f4af9406 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.587+0000 7fa652ffd640 1 -- 172.21.15.7:0/2269075724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c037070 con 0x7fa6540d1510 2024-09-15T23:37:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa652ffd640 1 -- 172.21.15.7:0/2269075724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63c032440 con 0x7fa6540d1510 2024-09-15T23:37:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa652ffd640 1 -- 172.21.15.7:0/2269075724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c032810 con 0x7fa6540d1510 2024-09-15T23:37:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa65a0af640 1 -- 172.21.15.7:0/2269075724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540d1510 msgr2=0x7fa6540b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa65a0af640 1 --2- 172.21.15.7:0/2269075724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540d1510 0x7fa6540b3c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0097a0 tx=0x7fa63c02a180 comp rx=0 tx=0).stop 2024-09-15T23:37:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa65a0af640 1 -- 172.21.15.7:0/2269075724 shutdown_connections 2024-09-15T23:37:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa65a0af640 1 --2- 172.21.15.7:0/2269075724 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540b5570 0x7fa6540b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa65a0af640 1 --2- 172.21.15.7:0/2269075724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540d1510 0x7fa6540b3c80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa65a0af640 1 -- 172.21.15.7:0/2269075724 >> 172.21.15.7:0/2269075724 conn(0x7fa65408bfd0 msgr2=0x7fa6540d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa65a0af640 1 -- 172.21.15.7:0/2269075724 shutdown_connections 2024-09-15T23:37:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.588+0000 7fa65a0af640 1 -- 172.21.15.7:0/2269075724 wait complete. 2024-09-15T23:37:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.589+0000 7fa65a0af640 1 Processor -- start 2024-09-15T23:37:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa65a0af640 1 -- start start 2024-09-15T23:37:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa65a0af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540b5570 0x7fa6541f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa65a0af640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540d1510 0x7fa65415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa65a0af640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa654161f20 0x7fa6540b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa65a0af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6540b38f0 con 0x7fa6540b5570 2024-09-15T23:37:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa65a0af640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa6540d0270 con 0x7fa654161f20 2024-09-15T23:37:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa65a0af640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fa6540b5c40 con 0x7fa6540d1510 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa6537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540b5570 0x7fa6541f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa6537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540b5570 0x7fa6541f8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53748/0 (socket says 172.21.15.7:53748) 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.590+0000 7fa6537fe640 1 -- 172.21.15.7:0/2590487504 learned_addr learned my addr 172.21.15.7:0/2590487504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa653fff640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa654161f20 0x7fa6540b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa64affd640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540d1510 0x7fa65415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa6537fe640 1 -- 172.21.15.7:0/2590487504 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540d1510 msgr2=0x7fa65415ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa6537fe640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540d1510 0x7fa65415ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa6537fe640 1 -- 172.21.15.7:0/2590487504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa654161f20 msgr2=0x7fa6540b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa6537fe640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa654161f20 0x7fa6540b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa6537fe640 1 -- 172.21.15.7:0/2590487504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65415b460 con 0x7fa6540b5570 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa653fff640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa654161f20 0x7fa6540b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa6537fe640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540b5570 0x7fa6541f8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa64400c400 tx=0x7fa64400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa650ff9640 1 -- 172.21.15.7:0/2590487504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644017070 con 0x7fa6540b5570 2024-09-15T23:37:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa650ff9640 1 -- 172.21.15.7:0/2590487504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa644005ce0 con 0x7fa6540b5570 2024-09-15T23:37:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65415b740 con 0x7fa6540b5570 2024-09-15T23:37:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa650ff9640 1 -- 172.21.15.7:0/2590487504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644007310 con 0x7fa6540b5570 2024-09-15T23:37:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.591+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65415bd00 con 0x7fa6540b5570 2024-09-15T23:37:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.592+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6540d0e00 con 0x7fa6540b5570 2024-09-15T23:37:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.593+0000 7fa650ff9640 1 -- 172.21.15.7:0/2590487504 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 90701+0+0 (secure 0 0 0) 0x7fa64401a070 con 0x7fa6540b5570 2024-09-15T23:37:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.593+0000 7fa650ff9640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fa63406c180 0x7fa63406e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.593+0000 7fa650ff9640 1 -- 172.21.15.7:0/2590487504 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fa64408a7a0 con 0x7fa6540b5570 2024-09-15T23:37:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.593+0000 7fa64affd640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fa63406c180 0x7fa63406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.594+0000 7fa64affd640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fa63406c180 0x7fa63406e640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa63c034000 tx=0x7fa63c02e000 comp rx=0 tx=0).ready entity=mgr.5067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.597+0000 7fa650ff9640 1 -- 172.21.15.7:0/2590487504 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa644058fc0 con 0x7fa6540b5570 2024-09-15T23:37:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.676+0000 7fa650ff9640 1 -- 172.21.15.7:0/2590487504 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 10) ==== 90966+0+0 (secure 0 0 0) 0x7fa644054f30 con 0x7fa6540b5570 2024-09-15T23:37:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.811+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 --> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7fa6541576c0 con 0x7fa63406c180 2024-09-15T23:37:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.870+0000 7fa650ff9640 1 -- 172.21.15.7:0/2590487504 <== mgr.5067 v2:172.21.15.7:6832/4033779377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa6541576c0 con 0x7fa63406c180 2024-09-15T23:37:32.871 INFO:teuthology.orchestra.run.smithi007.stdout:Schedule removed for path snap_test_dir1 2024-09-15T23:37:32.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fa63406c180 msgr2=0x7fa63406e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fa63406c180 0x7fa63406e640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa63c034000 tx=0x7fa63c02e000 comp rx=0 tx=0).stop 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540b5570 msgr2=0x7fa6541f8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540b5570 0x7fa6541f8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa64400c400 tx=0x7fa64400c8c0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 shutdown_connections 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fa63406c180 0x7fa63406e640 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa654161f20 0x7fa6540b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa6540d1510 0x7fa65415ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 --2- 172.21.15.7:0/2590487504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6540b5570 0x7fa6541f8b90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.872+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 >> 172.21.15.7:0/2590487504 conn(0x7fa65408bfd0 msgr2=0x7fa6540b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.873+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 shutdown_connections 2024-09-15T23:37:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:32.873+0000 7fa65a0af640 1 -- 172.21.15.7:0/2590487504 wait complete. 2024-09-15T23:37:32.884 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-09-15T23:37:32.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:32.884 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.customsnapkernel) 2024-09-15T23:37:32.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:32.902 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-09-15T23:37:32.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-09-15T23:37:33.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.103+0000 7f17086d0640 1 Processor -- start 2024-09-15T23:37:33.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.103+0000 7f17086d0640 1 -- start start 2024-09-15T23:37:33.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.104+0000 7f17086d0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f17000d0a00 0x7f17000d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.104+0000 7f17086d0640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f170015c860 0x7f17000cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.104+0000 7f17086d0640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1700160190 0x7f17001605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.104+0000 7f17086d0640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f17000cddc0 con 0x7f170015c860 2024-09-15T23:37:33.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.104+0000 7f17086d0640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f17000ccf30 con 0x7f1700160190 2024-09-15T23:37:33.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.104+0000 7f17086d0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f17000d1320 con 0x7f17000d0a00 2024-09-15T23:37:33.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1705c6b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f170015c860 0x7f17000cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1705c6b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f170015c860 0x7f17000cfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53790/0 (socket says 172.21.15.7:53790) 2024-09-15T23:37:33.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1705c6b640 1 -- 172.21.15.7:0/2071975222 learned_addr learned my addr 172.21.15.7:0/2071975222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1706c6d640 1 --2- 172.21.15.7:0/2071975222 >> v2:172.21.15.18:3300/0 conn(0x7f1700160190 0x7f17001605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f170646c640 1 --2- 172.21.15.7:0/2071975222 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f17000d0a00 0x7f17000d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1705c6b640 1 -- 172.21.15.7:0/2071975222 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f17000d0a00 msgr2=0x7f17000d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1705c6b640 1 --2- 172.21.15.7:0/2071975222 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f17000d0a00 0x7f17000d0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1705c6b640 1 -- 172.21.15.7:0/2071975222 >> v2:172.21.15.18:3300/0 conn(0x7f1700160190 msgr2=0x7f17001605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1705c6b640 1 --2- 172.21.15.7:0/2071975222 >> v2:172.21.15.18:3300/0 conn(0x7f1700160190 0x7f17001605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1705c6b640 1 -- 172.21.15.7:0/2071975222 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17000cd1b0 con 0x7f170015c860 2024-09-15T23:37:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f170646c640 1 --2- 172.21.15.7:0/2071975222 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f17000d0a00 0x7f17000d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:33.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.105+0000 7f1706c6d640 1 --2- 172.21.15.7:0/2071975222 >> v2:172.21.15.18:3300/0 conn(0x7f1700160190 0x7f17001605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:33.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f1705c6b640 1 --2- 172.21.15.7:0/2071975222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170015c860 0x7f17000cfff0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17000700d0 tx=0x7f16f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da19518e0010ad40 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:33.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f170546a640 1 -- 172.21.15.7:0/2071975222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0037070 con 0x7f170015c860 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f170546a640 1 -- 172.21.15.7:0/2071975222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f0032440 con 0x7f170015c860 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f170546a640 1 -- 172.21.15.7:0/2071975222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0032810 con 0x7f170015c860 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f17086d0640 1 -- 172.21.15.7:0/2071975222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170015c860 msgr2=0x7f17000cfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f17086d0640 1 --2- 172.21.15.7:0/2071975222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170015c860 0x7f17000cfff0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17000700d0 tx=0x7f16f002a1e0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f17086d0640 1 -- 172.21.15.7:0/2071975222 shutdown_connections 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f17086d0640 1 --2- 172.21.15.7:0/2071975222 >> v2:172.21.15.18:3300/0 conn(0x7f1700160190 0x7f17001605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f17086d0640 1 --2- 172.21.15.7:0/2071975222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170015c860 0x7f17000cfff0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f17086d0640 1 --2- 172.21.15.7:0/2071975222 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f17000d0a00 0x7f17000d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f17086d0640 1 -- 172.21.15.7:0/2071975222 >> 172.21.15.7:0/2071975222 conn(0x7f17000d2420 msgr2=0x7f17000d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.106+0000 7f17086d0640 1 -- 172.21.15.7:0/2071975222 shutdown_connections 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.107+0000 7f17086d0640 1 -- 172.21.15.7:0/2071975222 wait complete. 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.108+0000 7f17086d0640 1 Processor -- start 2024-09-15T23:37:33.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.108+0000 7f17086d0640 1 -- start start 2024-09-15T23:37:33.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.108+0000 7f17086d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17000d0a00 0x7f170015b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.108+0000 7f17086d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f170015c860 0x7f17000b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.108+0000 7f17086d0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1700160190 0x7f17000b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.108+0000 7f17086d0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17000cda50 con 0x7f17000d0a00 2024-09-15T23:37:33.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.108+0000 7f17086d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17000cd770 con 0x7f170015c860 2024-09-15T23:37:33.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.108+0000 7f17086d0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f170015dac0 con 0x7f1700160190 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.108+0000 7f170646c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17000d0a00 0x7f170015b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f170646c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17000d0a00 0x7f170015b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53798/0 (socket says 172.21.15.7:53798) 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f170646c640 1 -- 172.21.15.7:0/3480987225 learned_addr learned my addr 172.21.15.7:0/3480987225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f1706c6d640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1700160190 0x7f17000b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f1705c6b640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f170015c860 0x7f17000b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f170646c640 1 -- 172.21.15.7:0/3480987225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1700160190 msgr2=0x7f17000b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f170646c640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1700160190 0x7f17000b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f170646c640 1 -- 172.21.15.7:0/3480987225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f170015c860 msgr2=0x7f17000b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f170646c640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f170015c860 0x7f17000b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f170646c640 1 -- 172.21.15.7:0/3480987225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17000cb790 con 0x7f17000d0a00 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f170646c640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17000d0a00 0x7f170015b9e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c7f0 tx=0x7f16fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f16f6ffd640 1 -- 172.21.15.7:0/3480987225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc015070 con 0x7f17000d0a00 2024-09-15T23:37:33.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f16f6ffd640 1 -- 172.21.15.7:0/3480987225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc007820 con 0x7f17000d0a00 2024-09-15T23:37:33.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f16f6ffd640 1 -- 172.21.15.7:0/3480987225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc007c10 con 0x7f17000d0a00 2024-09-15T23:37:33.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17000cba70 con 0x7f17000d0a00 2024-09-15T23:37:33.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.109+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17000cc030 con 0x7f17000d0a00 2024-09-15T23:37:33.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.110+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17000d3250 con 0x7f17000d0a00 2024-09-15T23:37:33.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.111+0000 7f16f6ffd640 1 -- 172.21.15.7:0/3480987225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) ==== 90966+0+0 (secure 0 0 0) 0x7f16fc018070 con 0x7f17000d0a00 2024-09-15T23:37:33.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.111+0000 7f16f6ffd640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f16e406c420 0x7f16e406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.111+0000 7f16f6ffd640 1 -- 172.21.15.7:0/3480987225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f16fc08ba20 con 0x7f17000d0a00 2024-09-15T23:37:33.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.111+0000 7f1705c6b640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f16e406c420 0x7f16e406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.112+0000 7f1705c6b640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f16e406c420 0x7f16e406e8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16f0034000 tx=0x7f16f002e000 comp rx=0 tx=0).ready entity=mgr.5067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:33.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.116+0000 7f16f6ffd640 1 -- 172.21.15.7:0/3480987225 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f16fc059610 con 0x7f17000d0a00 2024-09-15T23:37:33.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.343+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f17000d2b70 con 0x7f17000d0a00 2024-09-15T23:37:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.675+0000 7f16f6ffd640 1 -- 172.21.15.7:0/3480987225 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v11) ==== 93+0+0 (secure 0 0 0) 0x7f16fc05d910 con 0x7f17000d0a00 2024-09-15T23:37:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.678+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f16e406c420 msgr2=0x7f16e406e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.678+0000 7f17086d0640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f16e406c420 0x7f16e406e8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16f0034000 tx=0x7f16f002e000 comp rx=0 tx=0).stop 2024-09-15T23:37:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.679+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17000d0a00 msgr2=0x7f170015b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.679+0000 7f17086d0640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17000d0a00 0x7f170015b9e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c7f0 tx=0x7f16fc00ccb0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.679+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 shutdown_connections 2024-09-15T23:37:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.679+0000 7f17086d0640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1700160190 0x7f17000b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.679+0000 7f17086d0640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f16e406c420 0x7f16e406e8e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.679+0000 7f17086d0640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f170015c860 0x7f17000b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.679+0000 7f17086d0640 1 --2- 172.21.15.7:0/3480987225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17000d0a00 0x7f170015b9e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.679+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 >> 172.21.15.7:0/3480987225 conn(0x7f17000d2420 msgr2=0x7f17000b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.679+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 shutdown_connections 2024-09-15T23:37:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.680+0000 7f17086d0640 1 -- 172.21.15.7:0/3480987225 wait complete. 2024-09-15T23:37:33.693 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-15T23:37:33.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-15T23:37:33.734 DEBUG:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-15T23:37:33.774 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-15T23:37:33.775 INFO:tasks.cephfs.mount:Terminating background process 2024-09-15T23:37:33.775 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:33.775 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:33.791 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:33.791 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:33.805 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:33.804+0000 7f7b80e90040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:37:33.807 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:33.806+0000 7f96a07d7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:37:33.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-15T23:37:33.849 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:33.848+0000 7f7b80e90040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:37:33.850 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:33.849+0000 7f96a07d7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:37:33.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.952+0000 7efd8ea0a640 1 Processor -- start 2024-09-15T23:37:33.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.952+0000 7efd8ea0a640 1 -- start start 2024-09-15T23:37:33.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.953+0000 7efd8ea0a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd880d24e0 0x7efd88162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.953+0000 7efd8ea0a640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efd8815e990 0x7efd880cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.953+0000 7efd8ea0a640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd880d2980 con 0x7efd8815e5b0 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.953+0000 7efd8ea0a640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efd880d2c40 con 0x7efd880d24e0 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.953+0000 7efd8ea0a640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7efd880d2f20 con 0x7efd8815e990 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.953+0000 7efd877fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd880d24e0 0x7efd88162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.953+0000 7efd877fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd880d24e0 0x7efd88162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53834/0 (socket says 172.21.15.7:53834) 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.953+0000 7efd877fe640 1 -- 172.21.15.7:0/2943027997 learned_addr learned my addr 172.21.15.7:0/2943027997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.953+0000 7efd8cfa7640 1 --2- 172.21.15.7:0/2943027997 >> v2:172.21.15.18:3300/0 conn(0x7efd8815e990 0x7efd880cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.954+0000 7efd877fe640 1 -- 172.21.15.7:0/2943027997 >> v2:172.21.15.18:3300/0 conn(0x7efd8815e990 msgr2=0x7efd880cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.954+0000 7efd877fe640 1 --2- 172.21.15.7:0/2943027997 >> v2:172.21.15.18:3300/0 conn(0x7efd8815e990 0x7efd880cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.954+0000 7efd877fe640 1 -- 172.21.15.7:0/2943027997 >> v1:172.21.15.18:6789/0 conn(0x7efd8815e5b0 legacy=0x7efd88162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.954+0000 7efd877fe640 1 -- 172.21.15.7:0/2943027997 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd880d3200 con 0x7efd880d24e0 2024-09-15T23:37:33.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.954+0000 7efd8cfa7640 1 --2- 172.21.15.7:0/2943027997 >> v2:172.21.15.18:3300/0 conn(0x7efd8815e990 0x7efd880cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:33.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.954+0000 7efd877fe640 1 --2- 172.21.15.7:0/2943027997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd880d24e0 0x7efd88162a60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009860 tx=0x7efd7c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735dca1157851c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:33.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.954+0000 7efd86ffd640 1 -- 172.21.15.7:0/2943027997 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c037070 con 0x7efd880d24e0 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.954+0000 7efd86ffd640 1 -- 172.21.15.7:0/2943027997 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c032440 con 0x7efd880d24e0 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.954+0000 7efd86ffd640 1 -- 172.21.15.7:0/2943027997 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c032810 con 0x7efd880d24e0 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.955+0000 7efd8ea0a640 1 -- 172.21.15.7:0/2943027997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd880d24e0 msgr2=0x7efd88162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.955+0000 7efd8ea0a640 1 --2- 172.21.15.7:0/2943027997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd880d24e0 0x7efd88162a60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009860 tx=0x7efd7c02a2a0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.955+0000 7efd8ea0a640 1 -- 172.21.15.7:0/2943027997 shutdown_connections 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.955+0000 7efd8ea0a640 1 --2- 172.21.15.7:0/2943027997 >> v2:172.21.15.18:3300/0 conn(0x7efd8815e990 0x7efd880cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.955+0000 7efd8ea0a640 1 --2- 172.21.15.7:0/2943027997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd880d24e0 0x7efd88162a60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.955+0000 7efd8ea0a640 1 -- 172.21.15.7:0/2943027997 >> 172.21.15.7:0/2943027997 conn(0x7efd8808bfd0 msgr2=0x7efd880d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.955+0000 7efd8ea0a640 1 -- 172.21.15.7:0/2943027997 shutdown_connections 2024-09-15T23:37:33.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.955+0000 7efd8ea0a640 1 -- 172.21.15.7:0/2943027997 wait complete. 2024-09-15T23:37:33.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.956+0000 7efd8ea0a640 1 Processor -- start 2024-09-15T23:37:33.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd8ea0a640 1 -- start start 2024-09-15T23:37:33.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd8ea0a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd880d24e0 0x7efd881618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd8ea0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8815e5b0 0x7efd880c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd8ea0a640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd8815e990 0x7efd880cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd8ea0a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd880b3b40 con 0x7efd8815e5b0 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd8ea0a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efd880b39c0 con 0x7efd880d24e0 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd8ea0a640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7efd88162fa0 con 0x7efd8815e990 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8815e5b0 0x7efd880c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8815e5b0 0x7efd880c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53836/0 (socket says 172.21.15.7:53836) 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd877fe640 1 -- 172.21.15.7:0/3415636614 learned_addr learned my addr 172.21.15.7:0/3415636614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd87fff640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd880d24e0 0x7efd881618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.957+0000 7efd8cfa7640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd8815e990 0x7efd880cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd877fe640 1 -- 172.21.15.7:0/3415636614 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd8815e990 msgr2=0x7efd880cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd877fe640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd8815e990 0x7efd880cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd877fe640 1 -- 172.21.15.7:0/3415636614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd880d24e0 msgr2=0x7efd881618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd877fe640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd880d24e0 0x7efd881618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd877fe640 1 -- 172.21.15.7:0/3415636614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd880c9500 con 0x7efd8815e5b0 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd877fe640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8815e5b0 0x7efd880c8a80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efd7c000c00 tx=0x7efd7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:33.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd84ff9640 1 -- 172.21.15.7:0/3415636614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c041020 con 0x7efd8815e5b0 2024-09-15T23:37:33.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd84ff9640 1 -- 172.21.15.7:0/3415636614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c0306c0 con 0x7efd8815e5b0 2024-09-15T23:37:33.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd880c97e0 con 0x7efd8815e5b0 2024-09-15T23:37:33.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd84ff9640 1 -- 172.21.15.7:0/3415636614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c037040 con 0x7efd8815e5b0 2024-09-15T23:37:33.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.958+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd880c9da0 con 0x7efd8815e5b0 2024-09-15T23:37:33.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.959+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd880d43c0 con 0x7efd8815e5b0 2024-09-15T23:37:33.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.960+0000 7efd84ff9640 1 -- 172.21.15.7:0/3415636614 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 90949+0+0 (secure 0 0 0) 0x7efd7c044020 con 0x7efd8815e5b0 2024-09-15T23:37:33.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.960+0000 7efd84ff9640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7efd7406c3d0 0x7efd7406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:33.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.960+0000 7efd87fff640 1 -- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7efd7406c3d0 msgr2=0x7efd7406e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:33.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.960+0000 7efd84ff9640 1 -- 172.21.15.7:0/3415636614 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7efd7c0ac320 con 0x7efd8815e5b0 2024-09-15T23:37:33.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.960+0000 7efd87fff640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7efd7406c3d0 0x7efd7406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:33.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:33.964+0000 7efd84ff9640 1 -- 172.21.15.7:0/3415636614 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efd880d43c0 con 0x7efd8815e5b0 2024-09-15T23:37:33.988 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:33.987+0000 7f96a07d7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:37:33.991 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:33.990+0000 7f7b80e90040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:37:34.039 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:34.038+0000 7f96a07d7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:37:34.042 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:34.042+0000 7f7b80e90040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:37:34.153 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:34.152+0000 7f96a07d7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:37:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.156+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efd880cb370 con 0x7efd8815e5b0 2024-09-15T23:37:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.156+0000 7efd84ff9640 1 -- 172.21.15.7:0/3415636614 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v40) ==== 80+0+344 (secure 0 0 0) 0x7efd880cb370 con 0x7efd8815e5b0 2024-09-15T23:37:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/571759858 2024-09-16T23:37:30.658739+0000 2024-09-15T23:37:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6832/3993537392 2024-09-16T23:37:30.658739+0000 2024-09-15T23:37:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/670254998 2024-09-16T23:37:30.658739+0000 2024-09-15T23:37:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/897775139 2024-09-16T23:37:30.658739+0000 2024-09-15T23:37:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6833/3993537392 2024-09-16T23:37:30.658739+0000 2024-09-15T23:37:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/991496469 2024-09-16T23:37:30.658739+0000 2024-09-15T23:37:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:listed 6 entries 2024-09-15T23:37:34.158 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:34.157+0000 7f7b80e90040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:37:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.158+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7efd7406c3d0 msgr2=0x7efd7406e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.158+0000 7efd8ea0a640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7efd7406c3d0 0x7efd7406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.158+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8815e5b0 msgr2=0x7efd880c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.158+0000 7efd8ea0a640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8815e5b0 0x7efd880c8a80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efd7c000c00 tx=0x7efd7c0046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.158+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 shutdown_connections 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.158+0000 7efd8ea0a640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efd8815e990 0x7efd880cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.158+0000 7efd8ea0a640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8815e5b0 0x7efd880c8a80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.158+0000 7efd8ea0a640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7efd7406c3d0 0x7efd7406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.158+0000 7efd8ea0a640 1 --2- 172.21.15.7:0/3415636614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd880d24e0 0x7efd881618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.159+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 >> 172.21.15.7:0/3415636614 conn(0x7efd8808bfd0 msgr2=0x7efd880b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.159+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 shutdown_connections 2024-09-15T23:37:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.159+0000 7efd8ea0a640 1 -- 172.21.15.7:0/3415636614 wait complete. 2024-09-15T23:37:34.170 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-15T23:37:34.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:34.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-15T23:37:34.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:37:34.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:34.200 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-15T23:37:34.251 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:34.250+0000 7f96a07d7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:37:34.256 INFO:tasks.cephfs.mount:Terminating background process 2024-09-15T23:37:34.256 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:34.256 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:34.259 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:34.259+0000 7f7b80e90040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:37:34.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1396 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-15T23:37:34.272 INFO:tasks.cephfs.mount:Terminating background process 2024-09-15T23:37:34.272 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:34.272 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:34.288 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056793,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1363 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249344k,nr_inodes=812336,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-15T23:37:34.289 INFO:tasks.cephfs.mount:Terminating background process 2024-09-15T23:37:34.289 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:34.289 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:34.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1396 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-15T23:37:34.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:34.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.430+0000 7f7df9fcd640 1 Processor -- start 2024-09-15T23:37:34.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.430+0000 7f7df9fcd640 1 -- start start 2024-09-15T23:37:34.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.431+0000 7f7df9fcd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7df415e840 0x7f7df40d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.431+0000 7f7df9fcd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df406dd80 con 0x7f7df40d0f90 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.431+0000 7f7df9fcd640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df40cf560 con 0x7f7df40d52e0 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.431+0000 7f7df9fcd640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7df40cfe90 con 0x7f7df415e840 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.431+0000 7f7df37fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7df415e840 0x7f7df40d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.432+0000 7f7df3fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7df40d0f90 0x7f7df40d24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39892/0 (socket says 172.21.15.7:39892) 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.432+0000 7f7df3fff640 1 -- 172.21.15.7:0/3987607648 learned_addr learned my addr 172.21.15.7:0/3987607648 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.432+0000 7f7df37fe640 1 -- 172.21.15.7:0/3987607648 >> v1:172.21.15.18:6789/0 conn(0x7f7df40d52e0 legacy=0x7f7df406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.432+0000 7f7df27fc640 1 -- 172.21.15.7:0/3987607648 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2698925432 0 0) 0x7f7df40cf560 con 0x7f7df40d52e0 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.432+0000 7f7df37fe640 1 -- 172.21.15.7:0/3987607648 >> v1:172.21.15.7:6789/0 conn(0x7f7df40d0f90 legacy=0x7f7df40d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.432+0000 7f7df37fe640 1 -- 172.21.15.7:0/3987607648 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df40cf840 con 0x7f7df415e840 2024-09-15T23:37:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.433+0000 7f7df37fe640 1 --2- 172.21.15.7:0/3987607648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df415e840 0x7f7df40d0320 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7de40097a0 tx=0x7f7de402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c188bb7769776c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.433+0000 7f7df27fc640 1 -- 172.21.15.7:0/3987607648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4037070 con 0x7f7df415e840 2024-09-15T23:37:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.433+0000 7f7df27fc640 1 -- 172.21.15.7:0/3987607648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de402adc0 con 0x7f7df415e840 2024-09-15T23:37:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.433+0000 7f7df27fc640 1 -- 172.21.15.7:0/3987607648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4032660 con 0x7f7df415e840 2024-09-15T23:37:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.434+0000 7f7df9fcd640 1 -- 172.21.15.7:0/3987607648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df415e840 msgr2=0x7f7df40d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.434+0000 7f7df9fcd640 1 --2- 172.21.15.7:0/3987607648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df415e840 0x7f7df40d0320 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7de40097a0 tx=0x7f7de402a130 comp rx=0 tx=0).stop 2024-09-15T23:37:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.434+0000 7f7df9fcd640 1 -- 172.21.15.7:0/3987607648 shutdown_connections 2024-09-15T23:37:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.434+0000 7f7df9fcd640 1 --2- 172.21.15.7:0/3987607648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df415e840 0x7f7df40d0320 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.434+0000 7f7df9fcd640 1 -- 172.21.15.7:0/3987607648 >> 172.21.15.7:0/3987607648 conn(0x7f7df408bfd0 msgr2=0x7f7df40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.434+0000 7f7df9fcd640 1 -- 172.21.15.7:0/3987607648 shutdown_connections 2024-09-15T23:37:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.434+0000 7f7df9fcd640 1 -- 172.21.15.7:0/3987607648 wait complete. 2024-09-15T23:37:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.435+0000 7f7df9fcd640 1 Processor -- start 2024-09-15T23:37:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.435+0000 7f7df9fcd640 1 -- start start 2024-09-15T23:37:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df9fcd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7df40d0f90 0x7f7df4161760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df9fcd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40d52e0 0x7f7df415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df9fcd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df415e840 0x7f7df415f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df9fcd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7df415ec20 con 0x7f7df40d52e0 2024-09-15T23:37:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df9fcd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7df406e030 con 0x7f7df415e840 2024-09-15T23:37:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df9fcd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7df40ca1b0 con 0x7f7df40d0f90 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40d52e0 0x7f7df415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40d52e0 0x7f7df415ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53846/0 (socket says 172.21.15.7:53846) 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df2ffd640 1 -- 172.21.15.7:0/2310572444 learned_addr learned my addr 172.21.15.7:0/2310572444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df37fe640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7df40d0f90 0x7f7df4161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df3fff640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df415e840 0x7f7df415f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df2ffd640 1 -- 172.21.15.7:0/2310572444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7df40d0f90 msgr2=0x7f7df4161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.436+0000 7f7df2ffd640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7df40d0f90 0x7f7df4161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.437+0000 7f7df2ffd640 1 -- 172.21.15.7:0/2310572444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df415e840 msgr2=0x7f7df415f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.437+0000 7f7df2ffd640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df415e840 0x7f7df415f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.437+0000 7f7df2ffd640 1 -- 172.21.15.7:0/2310572444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df40c95a0 con 0x7f7df40d52e0 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.437+0000 7f7df2ffd640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40d52e0 0x7f7df415ef30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7de800c480 tx=0x7f7de800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.437+0000 7f7ddbfff640 1 -- 172.21.15.7:0/2310572444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8017070 con 0x7f7df40d52e0 2024-09-15T23:37:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.437+0000 7f7ddbfff640 1 -- 172.21.15.7:0/2310572444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de8005480 con 0x7f7df40d52e0 2024-09-15T23:37:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.437+0000 7f7ddbfff640 1 -- 172.21.15.7:0/2310572444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8003880 con 0x7f7df40d52e0 2024-09-15T23:37:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.437+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df40c9880 con 0x7f7df40d52e0 2024-09-15T23:37:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.437+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df40c9e40 con 0x7f7df40d52e0 2024-09-15T23:37:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.438+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df40d08a0 con 0x7f7df40d52e0 2024-09-15T23:37:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.439+0000 7f7ddbfff640 1 -- 172.21.15.7:0/2310572444 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 90949+0+0 (secure 0 0 0) 0x7f7de801a070 con 0x7f7df40d52e0 2024-09-15T23:37:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.439+0000 7f7ddbfff640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f7de006c3d0 0x7f7de006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.439+0000 7f7df37fe640 1 -- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f7de006c3d0 msgr2=0x7f7de006e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.439+0000 7f7ddbfff640 1 -- 172.21.15.7:0/2310572444 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f7de808ab80 con 0x7f7df40d52e0 2024-09-15T23:37:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.439+0000 7f7df37fe640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f7de006c3d0 0x7f7de006e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.443+0000 7f7ddbfff640 1 -- 172.21.15.7:0/2310572444 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7de8058990 con 0x7f7df40d52e0 2024-09-15T23:37:34.531 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:34.530+0000 7f96a07d7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:37:34.544 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:34.543+0000 7f7b80e90040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:37:34.577 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:34.576+0000 7f96a07d7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:37:34.590 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:34.589+0000 7f7b80e90040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:37:34.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.640+0000 7f7df37fe640 1 -- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f7de006c3d0 msgr2=0x7f7de006e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:34.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.640+0000 7f7df37fe640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f7de006c3d0 0x7f7de006e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:37:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.655+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7df40d1cb0 con 0x7f7df40d52e0 2024-09-15T23:37:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.656+0000 7f7ddbfff640 1 -- 172.21.15.7:0/2310572444 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5214 (secure 0 0 0) 0x7f7de805de20 con 0x7f7df40d52e0 2024-09-15T23:37:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:34.657 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"btime":"2024-09-15T23:37:18:028501+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1101755994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6833","nonce":1101755994},{"type":"v1","addr":"172.21.15.18:6835","nonce":1101755994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6834/2816400413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2816400413},{"type":"v1","addr":"172.21.15.18:6834","nonce":2816400413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4827,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/809864756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":809864756},{"type":"v1","addr":"172.21.15.7:6836","nonce":809864756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:37:15.374444+0000","modified":"2024-09-15T23:37:17.368915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4830},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4830":{"gid":4830,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/1729414280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1729414280},{"type":"v1","addr":"172.21.15.7:6837","nonce":1729414280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4830,"qdb_cluster":[4830]},"id":2}]} 2024-09-15T23:37:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 15 2024-09-15T23:37:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f7de006c3d0 msgr2=0x7f7de006e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f7de006c3d0 0x7f7de006e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40d52e0 msgr2=0x7f7df415ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40d52e0 0x7f7df415ef30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7de800c480 tx=0x7f7de800c940 comp rx=0 tx=0).stop 2024-09-15T23:37:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 shutdown_connections 2024-09-15T23:37:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f7de006c3d0 0x7f7de006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df415e840 0x7f7df415f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40d52e0 0x7f7df415ef30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 --2- 172.21.15.7:0/2310572444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7df40d0f90 0x7f7df4161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 >> 172.21.15.7:0/2310572444 conn(0x7f7df408bfd0 msgr2=0x7f7df40b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 shutdown_connections 2024-09-15T23:37:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.658+0000 7f7df9fcd640 1 -- 172.21.15.7:0/2310572444 wait complete. 2024-09-15T23:37:34.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:34.699 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:34.698+0000 7f96a07d7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:37:34.715 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:34.715+0000 7f7b80e90040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:37:34.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.773+0000 7fb21e94b640 1 Processor -- start 2024-09-15T23:37:34.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.773+0000 7fb21e94b640 1 -- start start 2024-09-15T23:37:34.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.773+0000 7fb21e94b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb21806c320 0x7fb2180b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.773+0000 7fb21e94b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb218145940 0x7fb2180b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb21e94b640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2180ae3f0 con 0x7fb2181454e0 2024-09-15T23:37:34.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb21e94b640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb21809aec0 con 0x7fb21806c320 2024-09-15T23:37:34.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb21e94b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb2180b3840 con 0x7fb218145940 2024-09-15T23:37:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb2177fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb21806c320 0x7fb2180b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb2177fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb21806c320 0x7fb2180b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53856/0 (socket says 172.21.15.7:53856) 2024-09-15T23:37:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb2177fe640 1 -- 172.21.15.7:0/1104382130 learned_addr learned my addr 172.21.15.7:0/1104382130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb21cee8640 1 --2- 172.21.15.7:0/1104382130 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb218145940 0x7fb2180b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb2177fe640 1 -- 172.21.15.7:0/1104382130 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb218145940 msgr2=0x7fb2180b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb2177fe640 1 --2- 172.21.15.7:0/1104382130 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb218145940 0x7fb2180b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb2177fe640 1 -- 172.21.15.7:0/1104382130 >> v1:172.21.15.18:6789/0 conn(0x7fb2181454e0 legacy=0x7fb21806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.774+0000 7fb2177fe640 1 -- 172.21.15.7:0/1104382130 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21809abe0 con 0x7fb21806c320 2024-09-15T23:37:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.775+0000 7fb2177fe640 1 --2- 172.21.15.7:0/1104382130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 0x7fb2180b0650 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009870 tx=0x7fb20c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06a0757206a55e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.775+0000 7fb216ffd640 1 -- 172.21.15.7:0/1104382130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c037070 con 0x7fb21806c320 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.775+0000 7fb216ffd640 1 -- 172.21.15.7:0/1104382130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20c032440 con 0x7fb21806c320 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.775+0000 7fb216ffd640 1 -- 172.21.15.7:0/1104382130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c032810 con 0x7fb21806c320 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.775+0000 7fb21e94b640 1 -- 172.21.15.7:0/1104382130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 msgr2=0x7fb2180b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.775+0000 7fb21e94b640 1 --2- 172.21.15.7:0/1104382130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 0x7fb2180b0650 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009870 tx=0x7fb20c02a1c0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.776+0000 7fb21e94b640 1 -- 172.21.15.7:0/1104382130 shutdown_connections 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.776+0000 7fb21e94b640 1 --2- 172.21.15.7:0/1104382130 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb218145940 0x7fb2180b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.776+0000 7fb21e94b640 1 --2- 172.21.15.7:0/1104382130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 0x7fb2180b0650 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.776+0000 7fb21e94b640 1 -- 172.21.15.7:0/1104382130 >> 172.21.15.7:0/1104382130 conn(0x7fb218073490 msgr2=0x7fb2180b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.776+0000 7fb21e94b640 1 -- 172.21.15.7:0/1104382130 shutdown_connections 2024-09-15T23:37:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.776+0000 7fb21e94b640 1 -- 172.21.15.7:0/1104382130 wait complete. 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.777+0000 7fb21e94b640 1 Processor -- start 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.777+0000 7fb21e94b640 1 -- start start 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21e94b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 0x7fb2181f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21e94b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2181454e0 0x7fb2181f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21e94b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218145940 0x7fb218142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21e94b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2180b3e50 con 0x7fb21806c320 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21e94b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2180b3cd0 con 0x7fb218145940 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21e94b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb218145e90 con 0x7fb2181454e0 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb217fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 0x7fb2181f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2181454e0 0x7fb2181f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21cee8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218145940 0x7fb218142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb217fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 0x7fb2181f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53862/0 (socket says 172.21.15.7:53862) 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb217fff640 1 -- 172.21.15.7:0/3551672588 learned_addr learned my addr 172.21.15.7:0/3551672588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21cee8640 1 -- 172.21.15.7:0/3551672588 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2181454e0 msgr2=0x7fb2181f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21cee8640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2181454e0 0x7fb2181f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21cee8640 1 -- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 msgr2=0x7fb2181f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21cee8640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 0x7fb2181f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb21cee8640 1 -- 172.21.15.7:0/3551672588 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2181f5730 con 0x7fb218145940 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.778+0000 7fb2177fe640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2181454e0 0x7fb2181f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.779+0000 7fb21cee8640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218145940 0x7fb218142b60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb210007cf0 tx=0x7fb21000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.779+0000 7fb214ff9640 1 -- 172.21.15.7:0/3551672588 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21000edd0 con 0x7fb218145940 2024-09-15T23:37:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.779+0000 7fb214ff9640 1 -- 172.21.15.7:0/3551672588 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb210003990 con 0x7fb218145940 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.779+0000 7fb214ff9640 1 -- 172.21.15.7:0/3551672588 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210003d80 con 0x7fb218145940 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.779+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2181f5a10 con 0x7fb218145940 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.779+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2180b10f0 con 0x7fb218145940 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.780+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb21806bc10 con 0x7fb218145940 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.780+0000 7fb214ff9640 1 -- 172.21.15.7:0/3551672588 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 11) ==== 90949+0+0 (secure 0 0 0) 0x7fb21001b020 con 0x7fb218145940 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.781+0000 7fb214ff9640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fb20406c3d0 0x7fb20406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.781+0000 7fb217fff640 1 -- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fb20406c3d0 msgr2=0x7fb20406e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.781+0000 7fb214ff9640 1 -- 172.21.15.7:0/3551672588 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fb21008b810 con 0x7fb218145940 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.781+0000 7fb217fff640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fb20406c3d0 0x7fb20406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.785+0000 7fb214ff9640 1 -- 172.21.15.7:0/3551672588 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2100595a0 con 0x7fb218145940 2024-09-15T23:37:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.982+0000 7fb217fff640 1 -- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fb20406c3d0 msgr2=0x7fb20406e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.982+0000 7fb217fff640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fb20406c3d0 0x7fb20406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:37:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.996+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2180d1090 con 0x7fb218145940 2024-09-15T23:37:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.997+0000 7fb214ff9640 1 -- 172.21.15.7:0/3551672588 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5214 (secure 0 0 0) 0x7fb21005d8a0 con 0x7fb218145940 2024-09-15T23:37:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:34.998 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"btime":"2024-09-15T23:37:18:028501+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1101755994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6833","nonce":1101755994},{"type":"v1","addr":"172.21.15.18:6835","nonce":1101755994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6834/2816400413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2816400413},{"type":"v1","addr":"172.21.15.18:6834","nonce":2816400413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4827,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/809864756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":809864756},{"type":"v1","addr":"172.21.15.7:6836","nonce":809864756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:37:15.374444+0000","modified":"2024-09-15T23:37:17.368915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4830},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4830":{"gid":4830,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/1729414280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1729414280},{"type":"v1","addr":"172.21.15.7:6837","nonce":1729414280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4830,"qdb_cluster":[4830]},"id":2}]} 2024-09-15T23:37:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 15 2024-09-15T23:37:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fb20406c3d0 msgr2=0x7fb20406e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fb20406c3d0 0x7fb20406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218145940 msgr2=0x7fb218142b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218145940 0x7fb218142b60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb210007cf0 tx=0x7fb21000e4b0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 shutdown_connections 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fb20406c3d0 0x7fb20406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218145940 0x7fb218142b60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2181454e0 0x7fb2181f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 --2- 172.21.15.7:0/3551672588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21806c320 0x7fb2181f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:34.999+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 >> 172.21.15.7:0/3551672588 conn(0x7fb218073490 msgr2=0x7fb21815ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.000+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 shutdown_connections 2024-09-15T23:37:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.000+0000 7fb21e94b640 1 -- 172.21.15.7:0/3551672588 wait complete. 2024-09-15T23:37:35.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:37:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.115+0000 7f23cd911640 1 Processor -- start 2024-09-15T23:37:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.116+0000 7f23cd911640 1 -- start start 2024-09-15T23:37:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.116+0000 7f23cd911640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c806c320 0x7f23c80b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.116+0000 7f23cd911640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c80ae3f0 con 0x7f23c8145940 2024-09-15T23:37:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.116+0000 7f23cd911640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c809abe0 con 0x7f23c81454e0 2024-09-15T23:37:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.116+0000 7f23cd911640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f23c809aec0 con 0x7f23c806c320 2024-09-15T23:37:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.117+0000 7f23c6ffd640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f23c81454e0 0x7f23c806bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:40708/0 (socket says 172.21.15.7:40708) 2024-09-15T23:37:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.117+0000 7f23c6ffd640 1 -- 172.21.15.7:0/3497561636 learned_addr learned my addr 172.21.15.7:0/3497561636 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.117+0000 7f23c67fc640 1 --2- 172.21.15.7:0/3497561636 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c806c320 0x7f23c80b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.117+0000 7f23c5ffb640 1 -- 172.21.15.7:0/3497561636 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1438818669 0 0) 0x7f23c809abe0 con 0x7f23c81454e0 2024-09-15T23:37:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.117+0000 7f23c5ffb640 1 -- 172.21.15.7:0/3497561636 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23b0003520 con 0x7f23c81454e0 2024-09-15T23:37:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.117+0000 7f23c5ffb640 1 -- 172.21.15.7:0/3497561636 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467402789 0 0) 0x7f23c80ae3f0 con 0x7f23c8145940 2024-09-15T23:37:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.117+0000 7f23c67fc640 1 -- 172.21.15.7:0/3497561636 >> v1:172.21.15.18:6789/0 conn(0x7f23c81454e0 legacy=0x7f23c806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.117+0000 7f23c67fc640 1 -- 172.21.15.7:0/3497561636 >> v1:172.21.15.7:6789/0 conn(0x7f23c8145940 legacy=0x7f23c80b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.117+0000 7f23c67fc640 1 -- 172.21.15.7:0/3497561636 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c80b6730 con 0x7f23c806c320 2024-09-15T23:37:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.118+0000 7f23c67fc640 1 --2- 172.21.15.7:0/3497561636 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c806c320 0x7f23c80b0650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097d0 tx=0x7f23bc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28a1a229f4b50b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.118+0000 7f23c5ffb640 1 -- 172.21.15.7:0/3497561636 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc037070 con 0x7f23c806c320 2024-09-15T23:37:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.118+0000 7f23c5ffb640 1 -- 172.21.15.7:0/3497561636 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23bc032440 con 0x7f23c806c320 2024-09-15T23:37:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.118+0000 7f23c5ffb640 1 -- 172.21.15.7:0/3497561636 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0327f0 con 0x7f23c806c320 2024-09-15T23:37:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.119+0000 7f23cd911640 1 -- 172.21.15.7:0/3497561636 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c806c320 msgr2=0x7f23c80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.119+0000 7f23cd911640 1 --2- 172.21.15.7:0/3497561636 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c806c320 0x7f23c80b0650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097d0 tx=0x7f23bc02a180 comp rx=0 tx=0).stop 2024-09-15T23:37:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.119+0000 7f23cd911640 1 -- 172.21.15.7:0/3497561636 shutdown_connections 2024-09-15T23:37:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.119+0000 7f23cd911640 1 --2- 172.21.15.7:0/3497561636 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c806c320 0x7f23c80b0650 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.119+0000 7f23cd911640 1 -- 172.21.15.7:0/3497561636 >> 172.21.15.7:0/3497561636 conn(0x7f23c8073490 msgr2=0x7f23c80b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.119+0000 7f23cd911640 1 -- 172.21.15.7:0/3497561636 shutdown_connections 2024-09-15T23:37:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.119+0000 7f23cd911640 1 -- 172.21.15.7:0/3497561636 wait complete. 2024-09-15T23:37:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.120+0000 7f23cd911640 1 Processor -- start 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.120+0000 7f23cd911640 1 -- start start 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23cd911640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c806c320 0x7f23c815a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23cd911640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c81454e0 0x7f23c815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23cd911640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c8145940 0x7f23c80702a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23cd911640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8145e90 con 0x7f23c81454e0 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23cd911640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23c80b3840 con 0x7f23c806c320 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23cd911640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f23c8146010 con 0x7f23c8145940 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c81454e0 0x7f23c815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c81454e0 0x7f23c815b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53866/0 (socket says 172.21.15.7:53866) 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c67fc640 1 -- 172.21.15.7:0/1240236820 learned_addr learned my addr 172.21.15.7:0/1240236820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c6ffd640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c806c320 0x7f23c815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c77fe640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c8145940 0x7f23c80702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c67fc640 1 -- 172.21.15.7:0/1240236820 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c8145940 msgr2=0x7f23c80702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c67fc640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c8145940 0x7f23c80702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c67fc640 1 -- 172.21.15.7:0/1240236820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c806c320 msgr2=0x7f23c815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c67fc640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c806c320 0x7f23c815a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c67fc640 1 -- 172.21.15.7:0/1240236820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c80cf730 con 0x7f23c81454e0 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23c67fc640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c81454e0 0x7f23c815b640 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f23bc000c00 tx=0x7f23bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.121+0000 7f23affff640 1 -- 172.21.15.7:0/1240236820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc037050 con 0x7f23c81454e0 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.122+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c80cfa10 con 0x7f23c81454e0 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.122+0000 7f23affff640 1 -- 172.21.15.7:0/1240236820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23bc043070 con 0x7f23c81454e0 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.122+0000 7f23affff640 1 -- 172.21.15.7:0/1240236820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc032a30 con 0x7f23c81454e0 2024-09-15T23:37:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.122+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c80cffd0 con 0x7f23c81454e0 2024-09-15T23:37:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.123+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c809abe0 con 0x7f23c81454e0 2024-09-15T23:37:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.123+0000 7f23affff640 1 -- 172.21.15.7:0/1240236820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 90949+0+0 (secure 0 0 0) 0x7f23bc032bd0 con 0x7f23c81454e0 2024-09-15T23:37:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.124+0000 7f23affff640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f23b406c3d0 0x7f23b406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.124+0000 7f23c6ffd640 1 -- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f23b406c3d0 msgr2=0x7f23b406e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.124+0000 7f23affff640 1 -- 172.21.15.7:0/1240236820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f23bc0ac620 con 0x7f23c81454e0 2024-09-15T23:37:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.124+0000 7f23c6ffd640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f23b406c3d0 0x7f23b406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:35.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.128+0000 7f23affff640 1 -- 172.21.15.7:0/1240236820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f23bc07a280 con 0x7f23c81454e0 2024-09-15T23:37:35.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.306+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23c806faa0 con 0x7f23c81454e0 2024-09-15T23:37:35.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.307+0000 7f23affff640 1 -- 172.21.15.7:0/1240236820 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17273 (secure 0 0 0) 0x7f23bc07e580 con 0x7f23c81454e0 2024-09-15T23:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:37:30.658790+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:37:13.310698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-15T23:37:14.361918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/571759858":"2024-09-16T23:37:30.658739+0000","172.21.15.7:6832/3993537392":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/670254998":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/897775139":"2024-09-16T23:37:30.658739+0000","172.21.15.7:6833/3993537392":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/991496469":"2024-09-16T23:37:30.658739+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:37:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f23b406c3d0 msgr2=0x7f23b406e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f23b406c3d0 0x7f23b406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c81454e0 msgr2=0x7f23c815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c81454e0 0x7f23c815b640 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f23bc000c00 tx=0x7f23bc0046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 shutdown_connections 2024-09-15T23:37:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23c8145940 0x7f23c80702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c81454e0 0x7f23c815b640 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f23b406c3d0 0x7f23b406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 --2- 172.21.15.7:0/1240236820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c806c320 0x7f23c815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 >> 172.21.15.7:0/1240236820 conn(0x7f23c8073490 msgr2=0x7f23c806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:35.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 shutdown_connections 2024-09-15T23:37:35.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.310+0000 7f23cd911640 1 -- 172.21.15.7:0/1240236820 wait complete. 2024-09-15T23:37:35.322 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-15T23:37:35.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.426+0000 7f79203bd640 1 Processor -- start 2024-09-15T23:37:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.426+0000 7f79203bd640 1 -- start start 2024-09-15T23:37:35.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.427+0000 7f79203bd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791809b150 0x7f79180b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.427+0000 7f79203bd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f791809bcf0 con 0x7f791815fd10 2024-09-15T23:37:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.427+0000 7f79203bd640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79180d09f0 con 0x7f791809b780 2024-09-15T23:37:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.427+0000 7f79203bd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f79180d0cd0 con 0x7f791809b150 2024-09-15T23:37:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.427+0000 7f791e159640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f791815fd10 0x7f791809aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39912/0 (socket says 172.21.15.7:39912) 2024-09-15T23:37:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.427+0000 7f791e159640 1 -- 172.21.15.7:0/887878518 learned_addr learned my addr 172.21.15.7:0/887878518 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.427+0000 7f791d958640 1 --2- 172.21.15.7:0/887878518 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791809b150 0x7f79180b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.427+0000 7f791d157640 1 -- 172.21.15.7:0/887878518 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014373721 0 0) 0x7f791809bcf0 con 0x7f791815fd10 2024-09-15T23:37:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.427+0000 7f791d157640 1 -- 172.21.15.7:0/887878518 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7904003520 con 0x7f791815fd10 2024-09-15T23:37:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.428+0000 7f791d157640 1 -- 172.21.15.7:0/887878518 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 232252754 0 0) 0x7f79180d09f0 con 0x7f791809b780 2024-09-15T23:37:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.428+0000 7f791d157640 1 -- 172.21.15.7:0/887878518 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f791809bcf0 con 0x7f791809b780 2024-09-15T23:37:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.428+0000 7f791d958640 1 -- 172.21.15.7:0/887878518 >> v1:172.21.15.18:6789/0 conn(0x7f791809b780 legacy=0x7f79180cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.428+0000 7f791d958640 1 -- 172.21.15.7:0/887878518 >> v1:172.21.15.7:6789/0 conn(0x7f791815fd10 legacy=0x7f791809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.428+0000 7f791d958640 1 -- 172.21.15.7:0/887878518 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79180d0fb0 con 0x7f791809b150 2024-09-15T23:37:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.428+0000 7f791d958640 1 --2- 172.21.15.7:0/887878518 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791809b150 0x7f79180b30d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f790c0097d0 tx=0x7f790c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1bdbbc2b826d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f791d157640 1 -- 172.21.15.7:0/887878518 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c037070 con 0x7f791809b150 2024-09-15T23:37:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f791d157640 1 -- 172.21.15.7:0/887878518 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c032440 con 0x7f791809b150 2024-09-15T23:37:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f791d157640 1 -- 172.21.15.7:0/887878518 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c0327f0 con 0x7f791809b150 2024-09-15T23:37:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f79203bd640 1 -- 172.21.15.7:0/887878518 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791809b150 msgr2=0x7f79180b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f79203bd640 1 --2- 172.21.15.7:0/887878518 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791809b150 0x7f79180b30d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f790c0097d0 tx=0x7f790c02a210 comp rx=0 tx=0).stop 2024-09-15T23:37:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f79203bd640 1 -- 172.21.15.7:0/887878518 shutdown_connections 2024-09-15T23:37:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f79203bd640 1 --2- 172.21.15.7:0/887878518 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791809b150 0x7f79180b30d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f79203bd640 1 -- 172.21.15.7:0/887878518 >> 172.21.15.7:0/887878518 conn(0x7f79180d2410 msgr2=0x7f79180d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f79203bd640 1 -- 172.21.15.7:0/887878518 shutdown_connections 2024-09-15T23:37:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.429+0000 7f79203bd640 1 -- 172.21.15.7:0/887878518 wait complete. 2024-09-15T23:37:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.430+0000 7f79203bd640 1 Processor -- start 2024-09-15T23:37:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f79203bd640 1 -- start start 2024-09-15T23:37:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f79203bd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f791809b150 0x7f7918152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f79203bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f791809b780 0x7f79180b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f79203bd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791815fd10 0x7f79180b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f79203bd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79180cecf0 con 0x7f791809b780 2024-09-15T23:37:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f79203bd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f79180ceb70 con 0x7f791809b150 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f79203bd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f79180cd9e0 con 0x7f791815fd10 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f791d958640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f791809b780 0x7f79180b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f791d958640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f791809b780 0x7f79180b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53868/0 (socket says 172.21.15.7:53868) 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f791d958640 1 -- 172.21.15.7:0/65084830 learned_addr learned my addr 172.21.15.7:0/65084830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f791e159640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f791809b150 0x7f7918152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.431+0000 7f791e95a640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791815fd10 0x7f79180b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f791d958640 1 -- 172.21.15.7:0/65084830 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791815fd10 msgr2=0x7f79180b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f791d958640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791815fd10 0x7f79180b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f791d958640 1 -- 172.21.15.7:0/65084830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f791809b150 msgr2=0x7f7918152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f791d958640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f791809b150 0x7f7918152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f791d958640 1 -- 172.21.15.7:0/65084830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79180c8e20 con 0x7f791809b780 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f791d958640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f791809b780 0x7f79180b63c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f790c02a250 tx=0x7f790c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f790affd640 1 -- 172.21.15.7:0/65084830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c041020 con 0x7f791809b780 2024-09-15T23:37:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f790affd640 1 -- 172.21.15.7:0/65084830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c03d6e0 con 0x7f791809b780 2024-09-15T23:37:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79180c9100 con 0x7f791809b780 2024-09-15T23:37:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f790affd640 1 -- 172.21.15.7:0/65084830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c037040 con 0x7f791809b780 2024-09-15T23:37:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.432+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7918152bc0 con 0x7f791809b780 2024-09-15T23:37:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.433+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79180c5940 con 0x7f791809b780 2024-09-15T23:37:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.434+0000 7f790affd640 1 -- 172.21.15.7:0/65084830 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 90949+0+0 (secure 0 0 0) 0x7f790c044020 con 0x7f791809b780 2024-09-15T23:37:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.434+0000 7f790affd640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f78fc06c3d0 0x7f78fc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.434+0000 7f791e159640 1 -- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f78fc06c3d0 msgr2=0x7f78fc06e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.434+0000 7f790affd640 1 -- 172.21.15.7:0/65084830 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f790c0abe80 con 0x7f791809b780 2024-09-15T23:37:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.434+0000 7f791e159640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f78fc06c3d0 0x7f78fc06e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:35.440 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:35.440+0000 7f96a07d7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:37:35.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.443+0000 7f790affd640 1 -- 172.21.15.7:0/65084830 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f79180c5940 con 0x7f791809b780 2024-09-15T23:37:35.483 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:35.482+0000 7f7b80e90040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:37:35.487 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:35.486+0000 7f96a07d7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:37:35.530 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:35.529+0000 7f7b80e90040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:37:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.635+0000 7f791e159640 1 -- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f78fc06c3d0 msgr2=0x7f78fc06e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.635+0000 7f791e159640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f78fc06c3d0 0x7f78fc06e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:37:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.654+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f791809ae80 con 0x7f791809b780 2024-09-15T23:37:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.655+0000 7f790affd640 1 -- 172.21.15.7:0/65084830 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5214 (secure 0 0 0) 0x7f791809ae80 con 0x7f791809b780 2024-09-15T23:37:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:35.657 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"btime":"2024-09-15T23:37:18:028501+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1101755994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6833","nonce":1101755994},{"type":"v1","addr":"172.21.15.18:6835","nonce":1101755994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6834/2816400413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2816400413},{"type":"v1","addr":"172.21.15.18:6834","nonce":2816400413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4827,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/809864756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":809864756},{"type":"v1","addr":"172.21.15.7:6836","nonce":809864756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:37:15.374444+0000","modified":"2024-09-15T23:37:17.368915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4830},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4830":{"gid":4830,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/1729414280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1729414280},{"type":"v1","addr":"172.21.15.7:6837","nonce":1729414280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4830,"qdb_cluster":[4830]},"id":2}]} 2024-09-15T23:37:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 15 2024-09-15T23:37:35.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f78fc06c3d0 msgr2=0x7f78fc06e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:35.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f78fc06c3d0 0x7f78fc06e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f791809b780 msgr2=0x7f79180b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f791809b780 0x7f79180b63c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f790c02a250 tx=0x7f790c0046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 shutdown_connections 2024-09-15T23:37:35.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f791815fd10 0x7f79180b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f791809b780 0x7f79180b63c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f78fc06c3d0 0x7f78fc06e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 --2- 172.21.15.7:0/65084830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f791809b150 0x7f7918152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 >> 172.21.15.7:0/65084830 conn(0x7f79180d2410 msgr2=0x7f79180cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:35.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 shutdown_connections 2024-09-15T23:37:35.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.658+0000 7f79203bd640 1 -- 172.21.15.7:0/65084830 wait complete. 2024-09-15T23:37:35.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:37:35.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.772+0000 7fd774477640 1 Processor -- start 2024-09-15T23:37:35.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.772+0000 7fd774477640 1 -- start start 2024-09-15T23:37:35.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.773+0000 7fd774477640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd76c15e840 0x7fd76c0d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.773+0000 7fd774477640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd76c06dd80 con 0x7fd76c0d52e0 2024-09-15T23:37:35.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.773+0000 7fd774477640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd76c0cf560 con 0x7fd76c0d0f90 2024-09-15T23:37:35.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.773+0000 7fd774477640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd76c0cfe90 con 0x7fd76c15e840 2024-09-15T23:37:35.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.773+0000 7fd772213640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd76c15e840 0x7fd76c0d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.773+0000 7fd771a12640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd76c0d52e0 0x7fd76c06d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39920/0 (socket says 172.21.15.7:39920) 2024-09-15T23:37:35.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.773+0000 7fd771a12640 1 -- 172.21.15.7:0/2043386809 learned_addr learned my addr 172.21.15.7:0/2043386809 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:35.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.774+0000 7fd772213640 1 -- 172.21.15.7:0/2043386809 >> v1:172.21.15.18:6789/0 conn(0x7fd76c0d0f90 legacy=0x7fd76c0d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.774+0000 7fd772213640 1 -- 172.21.15.7:0/2043386809 >> v1:172.21.15.7:6789/0 conn(0x7fd76c0d52e0 legacy=0x7fd76c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.774+0000 7fd772213640 1 -- 172.21.15.7:0/2043386809 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c0cf840 con 0x7fd76c15e840 2024-09-15T23:37:35.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.774+0000 7fd772213640 1 --2- 172.21.15.7:0/2043386809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c15e840 0x7fd76c0d0320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd7680097a0 tx=0x7fd76802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1c4324ca8a177b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:35.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.774+0000 7fd771211640 1 -- 172.21.15.7:0/2043386809 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768037070 con 0x7fd76c15e840 2024-09-15T23:37:35.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.775+0000 7fd771211640 1 -- 172.21.15.7:0/2043386809 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd768032440 con 0x7fd76c15e840 2024-09-15T23:37:35.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.775+0000 7fd771211640 1 -- 172.21.15.7:0/2043386809 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768032810 con 0x7fd76c15e840 2024-09-15T23:37:35.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.775+0000 7fd774477640 1 -- 172.21.15.7:0/2043386809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c15e840 msgr2=0x7fd76c0d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.775+0000 7fd774477640 1 --2- 172.21.15.7:0/2043386809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c15e840 0x7fd76c0d0320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd7680097a0 tx=0x7fd76802a230 comp rx=0 tx=0).stop 2024-09-15T23:37:35.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.775+0000 7fd774477640 1 -- 172.21.15.7:0/2043386809 shutdown_connections 2024-09-15T23:37:35.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.775+0000 7fd774477640 1 --2- 172.21.15.7:0/2043386809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c15e840 0x7fd76c0d0320 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.775+0000 7fd774477640 1 -- 172.21.15.7:0/2043386809 >> 172.21.15.7:0/2043386809 conn(0x7fd76c08bfd0 msgr2=0x7fd76c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:35.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.775+0000 7fd774477640 1 -- 172.21.15.7:0/2043386809 shutdown_connections 2024-09-15T23:37:35.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.775+0000 7fd774477640 1 -- 172.21.15.7:0/2043386809 wait complete. 2024-09-15T23:37:35.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.777+0000 7fd774477640 1 Processor -- start 2024-09-15T23:37:35.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.777+0000 7fd774477640 1 -- start start 2024-09-15T23:37:35.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.777+0000 7fd774477640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c0d0f90 0x7fd76c15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.777+0000 7fd774477640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd76c0d52e0 0x7fd76c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.777+0000 7fd774477640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd76c15e840 0x7fd76c0b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.777+0000 7fd774477640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c15ec20 con 0x7fd76c0d0f90 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.777+0000 7fd774477640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c06e030 con 0x7fd76c0d52e0 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.777+0000 7fd774477640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fd76c15b0d0 con 0x7fd76c15e840 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772213640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c0d0f90 0x7fd76c15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772213640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c0d0f90 0x7fd76c15ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53880/0 (socket says 172.21.15.7:53880) 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772213640 1 -- 172.21.15.7:0/3543120161 learned_addr learned my addr 172.21.15.7:0/3543120161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd771a12640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd76c0d52e0 0x7fd76c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772a14640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd76c15e840 0x7fd76c0b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:35.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772213640 1 -- 172.21.15.7:0/3543120161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd76c15e840 msgr2=0x7fd76c0b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772213640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd76c15e840 0x7fd76c0b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772213640 1 -- 172.21.15.7:0/3543120161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd76c0d52e0 msgr2=0x7fd76c0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772213640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd76c0d52e0 0x7fd76c0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772213640 1 -- 172.21.15.7:0/3543120161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c1f4d00 con 0x7fd76c0d0f90 2024-09-15T23:37:35.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd772213640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c0d0f90 0x7fd76c15ab90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd7680096f0 tx=0x7fd7680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:35.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd75effd640 1 -- 172.21.15.7:0/3543120161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768037040 con 0x7fd76c0d0f90 2024-09-15T23:37:35.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd75effd640 1 -- 172.21.15.7:0/3543120161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7680307b0 con 0x7fd76c0d0f90 2024-09-15T23:37:35.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c1f4fe0 con 0x7fd76c0d0f90 2024-09-15T23:37:35.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd75effd640 1 -- 172.21.15.7:0/3543120161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768030b80 con 0x7fd76c0d0f90 2024-09-15T23:37:35.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.778+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1f55a0 con 0x7fd76c0d0f90 2024-09-15T23:37:35.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.779+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c0d08a0 con 0x7fd76c0d0f90 2024-09-15T23:37:35.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.780+0000 7fd75effd640 1 -- 172.21.15.7:0/3543120161 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 90949+0+0 (secure 0 0 0) 0x7fd768044020 con 0x7fd76c0d0f90 2024-09-15T23:37:35.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.780+0000 7fd75effd640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fd75006c3d0 0x7fd75006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:35.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.780+0000 7fd771a12640 1 -- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fd75006c3d0 msgr2=0x7fd75006e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:35.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.781+0000 7fd75effd640 1 -- 172.21.15.7:0/3543120161 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fd7680ab9a0 con 0x7fd76c0d0f90 2024-09-15T23:37:35.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.781+0000 7fd771a12640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fd75006c3d0 0x7fd75006e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.785+0000 7fd75effd640 1 -- 172.21.15.7:0/3543120161 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd76804b020 con 0x7fd76c0d0f90 2024-09-15T23:37:35.816 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:35.815+0000 7f96a07d7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:37:35.868 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:35.867+0000 7f7b80e90040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:37:35.903 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:35.902+0000 7f96a07d7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:37:35.957 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:35.956+0000 7f7b80e90040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:37:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.982+0000 7fd771a12640 1 -- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fd75006c3d0 msgr2=0x7fd75006e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.982+0000 7fd771a12640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fd75006c3d0 0x7fd75006e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:37:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.993+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd76c0cbe80 con 0x7fd76c0d0f90 2024-09-15T23:37:35.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.994+0000 7fd75effd640 1 -- 172.21.15.7:0/3543120161 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5214 (secure 0 0 0) 0x7fd768032e20 con 0x7fd76c0d0f90 2024-09-15T23:37:35.995 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:35.995 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"btime":"2024-09-15T23:37:18:028501+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1101755994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6833","nonce":1101755994},{"type":"v1","addr":"172.21.15.18:6835","nonce":1101755994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6834/2816400413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2816400413},{"type":"v1","addr":"172.21.15.18:6834","nonce":2816400413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4827,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/809864756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":809864756},{"type":"v1","addr":"172.21.15.7:6836","nonce":809864756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T23:37:15.374444+0000","modified":"2024-09-15T23:37:17.368915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4830},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4830":{"gid":4830,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/1729414280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1729414280},{"type":"v1","addr":"172.21.15.7:6837","nonce":1729414280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4830,"qdb_cluster":[4830]},"id":2}]} 2024-09-15T23:37:35.995 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 15 2024-09-15T23:37:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fd75006c3d0 msgr2=0x7fd75006e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fd75006c3d0 0x7fd75006e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c0d0f90 msgr2=0x7fd76c15ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c0d0f90 0x7fd76c15ab90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd7680096f0 tx=0x7fd7680046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 shutdown_connections 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd76c15e840 0x7fd76c0b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fd75006c3d0 0x7fd75006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd76c0d52e0 0x7fd76c0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 --2- 172.21.15.7:0/3543120161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd76c0d0f90 0x7fd76c15ab90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 >> 172.21.15.7:0/3543120161 conn(0x7fd76c08bfd0 msgr2=0x7fd76c0b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 shutdown_connections 2024-09-15T23:37:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:35.996+0000 7fd774477640 1 -- 172.21.15.7:0/3543120161 wait complete. 2024-09-15T23:37:36.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:37:36.015 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.014+0000 7f96a07d7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:37:36.059 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.058+0000 7f96a07d7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:37:36.072 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.071+0000 7f7b80e90040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:37:36.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.110+0000 7fe81269b640 1 Processor -- start 2024-09-15T23:37:36.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.110+0000 7fe81269b640 1 -- start start 2024-09-15T23:37:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe81269b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe80c0d52e0 0x7fe80c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe81269b640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe80c06dbb0 0x7fe80c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe81269b640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80c0d0a30 con 0x7fe80c15e840 2024-09-15T23:37:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe81269b640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe80c0cfe90 con 0x7fe80c0d52e0 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe81269b640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe80c0cf560 con 0x7fe80c06dbb0 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe80b7fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe80c0d52e0 0x7fe80c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe80b7fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe80c0d52e0 0x7fe80c06d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53894/0 (socket says 172.21.15.7:53894) 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe80b7fe640 1 -- 172.21.15.7:0/2531313416 learned_addr learned my addr 172.21.15.7:0/2531313416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe810c38640 1 --2- 172.21.15.7:0/2531313416 >> v2:172.21.15.18:3300/0 conn(0x7fe80c06dbb0 0x7fe80c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe80b7fe640 1 -- 172.21.15.7:0/2531313416 >> v2:172.21.15.18:3300/0 conn(0x7fe80c06dbb0 msgr2=0x7fe80c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe80b7fe640 1 --2- 172.21.15.7:0/2531313416 >> v2:172.21.15.18:3300/0 conn(0x7fe80c06dbb0 0x7fe80c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe80b7fe640 1 -- 172.21.15.7:0/2531313416 >> v1:172.21.15.18:6789/0 conn(0x7fe80c15e840 legacy=0x7fe80c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe80affd640 1 -- 172.21.15.7:0/2531313416 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032783916 0 0) 0x7fe80c0d0a30 con 0x7fe80c15e840 2024-09-15T23:37:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.111+0000 7fe80b7fe640 1 -- 172.21.15.7:0/2531313416 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c0cf840 con 0x7fe80c0d52e0 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.112+0000 7fe80b7fe640 1 --2- 172.21.15.7:0/2531313416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 0x7fe80c06d670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0cb620 tx=0x7fe80002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293ce6dd95bec220 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.112+0000 7fe80affd640 1 -- 172.21.15.7:0/2531313416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800004030 con 0x7fe80c0d52e0 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.112+0000 7fe80affd640 1 -- 172.21.15.7:0/2531313416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8000041d0 con 0x7fe80c0d52e0 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.112+0000 7fe80affd640 1 -- 172.21.15.7:0/2531313416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800003610 con 0x7fe80c0d52e0 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.113+0000 7fe81269b640 1 -- 172.21.15.7:0/2531313416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 msgr2=0x7fe80c06d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.113+0000 7fe81269b640 1 --2- 172.21.15.7:0/2531313416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 0x7fe80c06d670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0cb620 tx=0x7fe80002a270 comp rx=0 tx=0).stop 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.113+0000 7fe81269b640 1 -- 172.21.15.7:0/2531313416 shutdown_connections 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.113+0000 7fe81269b640 1 --2- 172.21.15.7:0/2531313416 >> v2:172.21.15.18:3300/0 conn(0x7fe80c06dbb0 0x7fe80c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.113+0000 7fe81269b640 1 --2- 172.21.15.7:0/2531313416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 0x7fe80c06d670 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.113+0000 7fe81269b640 1 -- 172.21.15.7:0/2531313416 >> 172.21.15.7:0/2531313416 conn(0x7fe80c08bfd0 msgr2=0x7fe80c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.113+0000 7fe81269b640 1 -- 172.21.15.7:0/2531313416 shutdown_connections 2024-09-15T23:37:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.113+0000 7fe81269b640 1 -- 172.21.15.7:0/2531313416 wait complete. 2024-09-15T23:37:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.115+0000 7fe81269b640 1 Processor -- start 2024-09-15T23:37:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.115+0000 7fe81269b640 1 -- start start 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe81269b640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe80c06dbb0 0x7fe80c15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe81269b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 0x7fe80c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe81269b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe80c15e840 0x7fe80c1610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe81269b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c15ec20 con 0x7fe80c0d52e0 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe81269b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c06e030 con 0x7fe80c15e840 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe81269b640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c14b6c0 con 0x7fe80c06dbb0 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 0x7fe80c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 0x7fe80c0b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53896/0 (socket says 172.21.15.7:53896) 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80b7fe640 1 -- 172.21.15.7:0/4164422505 learned_addr learned my addr 172.21.15.7:0/4164422505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe810c38640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe80c15e840 0x7fe80c1610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80bfff640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe80c06dbb0 0x7fe80c15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80b7fe640 1 -- 172.21.15.7:0/4164422505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe80c06dbb0 msgr2=0x7fe80c15f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80b7fe640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe80c06dbb0 0x7fe80c15f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80b7fe640 1 -- 172.21.15.7:0/4164422505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe80c15e840 msgr2=0x7fe80c1610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80b7fe640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe80c15e840 0x7fe80c1610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80b7fe640 1 -- 172.21.15.7:0/4164422505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c0b5a50 con 0x7fe80c0d52e0 2024-09-15T23:37:36.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe80b7fe640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 0x7fe80c0b5070 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe800034040 tx=0x7fe8000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:36.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.117+0000 7fe808ff9640 1 -- 172.21.15.7:0/4164422505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800041020 con 0x7fe80c0d52e0 2024-09-15T23:37:36.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.118+0000 7fe808ff9640 1 -- 172.21.15.7:0/4164422505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe800003c50 con 0x7fe80c0d52e0 2024-09-15T23:37:36.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.118+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c0b5d30 con 0x7fe80c0d52e0 2024-09-15T23:37:36.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.118+0000 7fe808ff9640 1 -- 172.21.15.7:0/4164422505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800004030 con 0x7fe80c0d52e0 2024-09-15T23:37:36.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.118+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c0b62f0 con 0x7fe80c0d52e0 2024-09-15T23:37:36.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.119+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c06d670 con 0x7fe80c0d52e0 2024-09-15T23:37:36.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.119+0000 7fe808ff9640 1 -- 172.21.15.7:0/4164422505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 90949+0+0 (secure 0 0 0) 0x7fe800044020 con 0x7fe80c0d52e0 2024-09-15T23:37:36.122 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.120+0000 7f7b80e90040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:37:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.120+0000 7fe808ff9640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fe7f806c3d0 0x7fe7f806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.120+0000 7fe80bfff640 1 -- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fe7f806c3d0 msgr2=0x7fe7f806e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.120+0000 7fe808ff9640 1 -- 172.21.15.7:0/4164422505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fe8000ac1c0 con 0x7fe80c0d52e0 2024-09-15T23:37:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.120+0000 7fe80bfff640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fe7f806c3d0 0x7fe7f806e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.124+0000 7fe808ff9640 1 -- 172.21.15.7:0/4164422505 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe80004b030 con 0x7fe80c0d52e0 2024-09-15T23:37:36.176 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.175+0000 7f96a07d7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:37:36.229 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.228+0000 7f96a07d7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:37:36.239 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.238+0000 7f7b80e90040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:37:36.272 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.271+0000 7f96a07d7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:37:36.294 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.293+0000 7f7b80e90040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:37:36.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.304+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe80c0d1cb0 con 0x7fe80c0d52e0 2024-09-15T23:37:36.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.305+0000 7fe808ff9640 1 -- 172.21.15.7:0/4164422505 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17273 (secure 0 0 0) 0x7fe800079fd0 con 0x7fe80c0d52e0 2024-09-15T23:37:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:36.306 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:37:30.658790+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:37:13.310698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-15T23:37:14.361918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/571759858":"2024-09-16T23:37:30.658739+0000","172.21.15.7:6832/3993537392":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/670254998":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/897775139":"2024-09-16T23:37:30.658739+0000","172.21.15.7:6833/3993537392":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/991496469":"2024-09-16T23:37:30.658739+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:37:36.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fe7f806c3d0 msgr2=0x7fe7f806e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:36.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fe7f806c3d0 0x7fe7f806e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 msgr2=0x7fe80c0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 0x7fe80c0b5070 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe800034040 tx=0x7fe8000046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 shutdown_connections 2024-09-15T23:37:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fe7f806c3d0 0x7fe7f806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe80c15e840 0x7fe80c1610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe80c0d52e0 0x7fe80c0b5070 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 --2- 172.21.15.7:0/4164422505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe80c06dbb0 0x7fe80c15f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.307+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 >> 172.21.15.7:0/4164422505 conn(0x7fe80c08bfd0 msgr2=0x7fe80c06cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.308+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 shutdown_connections 2024-09-15T23:37:36.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.308+0000 7fe81269b640 1 -- 172.21.15.7:0/4164422505 wait complete. 2024-09-15T23:37:36.320 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-15T23:37:36.339 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.339+0000 7f7b80e90040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:37:36.390 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.389+0000 7f96a07d7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:37:36.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.423+0000 7fad7859f640 1 Processor -- start 2024-09-15T23:37:36.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.423+0000 7fad7859f640 1 -- start start 2024-09-15T23:37:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7859f640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad7015e5b0 0x7fad70162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7859f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad700d24e0 0x7fad70162890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7859f640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fad7015e990 0x7fad70162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7859f640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fad700d2980 con 0x7fad7015e5b0 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7859f640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fad700d2c40 con 0x7fad7015e990 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7859f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fad700d2f20 con 0x7fad700d24e0 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7633b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad7015e5b0 0x7fad70162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7633b640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad7015e5b0 0x7fad70162350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53910/0 (socket says 172.21.15.7:53910) 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7633b640 1 -- 172.21.15.7:0/1563380898 learned_addr learned my addr 172.21.15.7:0/1563380898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad76b3c640 1 --2- 172.21.15.7:0/1563380898 >> v2:172.21.15.18:3300/0 conn(0x7fad7015e990 0x7fad70162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad75b3a640 1 --2- 172.21.15.7:0/1563380898 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad700d24e0 0x7fad70162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7633b640 1 -- 172.21.15.7:0/1563380898 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad700d24e0 msgr2=0x7fad70162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7633b640 1 --2- 172.21.15.7:0/1563380898 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad700d24e0 0x7fad70162890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7633b640 1 -- 172.21.15.7:0/1563380898 >> v2:172.21.15.18:3300/0 conn(0x7fad7015e990 msgr2=0x7fad70162dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.424+0000 7fad7633b640 1 --2- 172.21.15.7:0/1563380898 >> v2:172.21.15.18:3300/0 conn(0x7fad7015e990 0x7fad70162dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.425+0000 7fad7633b640 1 -- 172.21.15.7:0/1563380898 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad700d3200 con 0x7fad7015e5b0 2024-09-15T23:37:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.425+0000 7fad7633b640 1 --2- 172.21.15.7:0/1563380898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7015e5b0 0x7fad70162350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fad700b3790 tx=0x7fad6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd94f9ac090429 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.425+0000 7fad75339640 1 -- 172.21.15.7:0/1563380898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c037070 con 0x7fad7015e5b0 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.425+0000 7fad75339640 1 -- 172.21.15.7:0/1563380898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c032440 con 0x7fad7015e5b0 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.425+0000 7fad75339640 1 -- 172.21.15.7:0/1563380898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c032810 con 0x7fad7015e5b0 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.426+0000 7fad7859f640 1 -- 172.21.15.7:0/1563380898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7015e5b0 msgr2=0x7fad70162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.426+0000 7fad7859f640 1 --2- 172.21.15.7:0/1563380898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7015e5b0 0x7fad70162350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fad700b3790 tx=0x7fad6c02a1e0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.426+0000 7fad7859f640 1 -- 172.21.15.7:0/1563380898 shutdown_connections 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.426+0000 7fad7859f640 1 --2- 172.21.15.7:0/1563380898 >> v2:172.21.15.18:3300/0 conn(0x7fad7015e990 0x7fad70162dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.426+0000 7fad7859f640 1 --2- 172.21.15.7:0/1563380898 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad700d24e0 0x7fad70162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.426+0000 7fad7859f640 1 --2- 172.21.15.7:0/1563380898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7015e5b0 0x7fad70162350 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.426+0000 7fad7859f640 1 -- 172.21.15.7:0/1563380898 >> 172.21.15.7:0/1563380898 conn(0x7fad7008bfd0 msgr2=0x7fad700d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.426+0000 7fad7859f640 1 -- 172.21.15.7:0/1563380898 shutdown_connections 2024-09-15T23:37:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.426+0000 7fad7859f640 1 -- 172.21.15.7:0/1563380898 wait complete. 2024-09-15T23:37:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.427+0000 7fad7859f640 1 Processor -- start 2024-09-15T23:37:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.427+0000 7fad7859f640 1 -- start start 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7859f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad700d24e0 0x7fad7014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7859f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad7015e5b0 0x7fad700cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7859f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7015e990 0x7fad700cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7859f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad700b3b40 con 0x7fad7015e990 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7859f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fad700b39c0 con 0x7fad700d24e0 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7859f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fad700d0fa0 con 0x7fad7015e5b0 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7633b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad700d24e0 0x7fad7014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad76b3c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7015e990 0x7fad700cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7633b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad700d24e0 0x7fad7014fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:48804/0 (socket says 172.21.15.7:48804) 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7633b640 1 -- 172.21.15.7:0/3892895109 learned_addr learned my addr 172.21.15.7:0/3892895109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad75b3a640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad7015e5b0 0x7fad700cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7633b640 1 -- 172.21.15.7:0/3892895109 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad7015e5b0 msgr2=0x7fad700cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.428+0000 7fad7633b640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad7015e5b0 0x7fad700cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.429+0000 7fad7633b640 1 -- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7015e990 msgr2=0x7fad700cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.429+0000 7fad7633b640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7015e990 0x7fad700cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.429+0000 7fad7633b640 1 -- 172.21.15.7:0/3892895109 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7015dd90 con 0x7fad700d24e0 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.429+0000 7fad7633b640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad700d24e0 0x7fad7014fa30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fad6c02a220 tx=0x7fad6c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.429+0000 7fad66ffd640 1 -- 172.21.15.7:0/3892895109 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c041020 con 0x7fad700d24e0 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.429+0000 7fad66ffd640 1 -- 172.21.15.7:0/3892895109 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c03d620 con 0x7fad700d24e0 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.429+0000 7fad66ffd640 1 -- 172.21.15.7:0/3892895109 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c037040 con 0x7fad700d24e0 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.429+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad700cb8c0 con 0x7fad700d24e0 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.429+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad700cbe80 con 0x7fad700d24e0 2024-09-15T23:37:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.430+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad700d13d0 con 0x7fad700d24e0 2024-09-15T23:37:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.431+0000 7fad66ffd640 1 -- 172.21.15.7:0/3892895109 <== mon.1 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 11) ==== 90949+0+0 (secure 0 0 0) 0x7fad6c044020 con 0x7fad700d24e0 2024-09-15T23:37:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.431+0000 7fad66ffd640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fad5406c3d0 0x7fad5406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.431+0000 7fad75b3a640 1 -- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fad5406c3d0 msgr2=0x7fad5406e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.431+0000 7fad66ffd640 1 -- 172.21.15.7:0/3892895109 <== mon.1 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fad6c0abe40 con 0x7fad700d24e0 2024-09-15T23:37:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.431+0000 7fad75b3a640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fad5406c3d0 0x7fad5406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:37:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.435+0000 7fad66ffd640 1 -- 172.21.15.7:0/3892895109 <== mon.1 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad6c04b030 con 0x7fad700d24e0 2024-09-15T23:37:36.441 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.440+0000 7f96a07d7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:37:36.461 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.460+0000 7f7b80e90040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:37:36.494 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.493+0000 7f96a07d7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:37:36.512 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.511+0000 7f7b80e90040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:37:36.549 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.548+0000 7f96a07d7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:37:36.563 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.562+0000 7f7b80e90040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:37:36.593 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.592+0000 7f96a07d7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:37:36.618 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.617+0000 7f7b80e90040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:37:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.632+0000 7fad75b3a640 1 -- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fad5406c3d0 msgr2=0x7fad5406e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6832/4033779377 2024-09-15T23:37:36.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.632+0000 7fad75b3a640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fad5406c3d0 0x7fad5406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:37:36.637 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:36.636+0000 7f96a07d7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:37:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.645+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fad700ba560 con 0x7fad700d24e0 2024-09-15T23:37:36.647 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-15T23:37:36.646+0000 7f6dee804640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T23:37:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.650+0000 7fad66ffd640 1 -- 172.21.15.7:0/3892895109 <== mon.1 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) ==== 161+0+0 (secure 0 0 0) 0x7fad6c079b40 con 0x7fad700d24e0 2024-09-15T23:37:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-15T23:37:36.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fad5406c3d0 msgr2=0x7fad5406e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:37:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fad5406c3d0 0x7fad5406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad700d24e0 msgr2=0x7fad7014fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad700d24e0 0x7fad7014fa30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fad6c02a220 tx=0x7fad6c0046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 shutdown_connections 2024-09-15T23:37:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7015e990 0x7fad700cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fad7015e5b0 0x7fad700cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7fad5406c3d0 0x7fad5406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 --2- 172.21.15.7:0/3892895109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad700d24e0 0x7fad7014fa30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 >> 172.21.15.7:0/3892895109 conn(0x7fad7008bfd0 msgr2=0x7fad700b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:36.655 INFO:tasks.ceph.mds.a.smithi007.stderr: -3778> 2024-09-15T23:37:20.997+0000 7f9413af7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T23:37:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 shutdown_connections 2024-09-15T23:37:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.653+0000 7fad7859f640 1 -- 172.21.15.7:0/3892895109 wait complete. 2024-09-15T23:37:36.665 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.665+0000 7f7b80e90040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:37:36.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-15T23:37:36.686 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-15T23:37:36.714 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:36.713+0000 7f7b80e90040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:37:36.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.791+0000 7f80d7853640 1 Processor -- start 2024-09-15T23:37:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.791+0000 7f80d7853640 1 -- start start 2024-09-15T23:37:36.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.792+0000 7f80d7853640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f80d00d24e0 0x7f80d0162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.792+0000 7f80d7853640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80d015e990 0x7f80d00cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.792+0000 7f80d7853640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d00d2980 con 0x7f80d015e5b0 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.792+0000 7f80d7853640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f80d00d2c40 con 0x7f80d015e990 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.792+0000 7f80d7853640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f80d00d2f20 con 0x7f80d00d24e0 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.792+0000 7f80d5df0640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80d015e990 0x7f80d00cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.792+0000 7f80d5df0640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80d015e990 0x7f80d00cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53940/0 (socket says 172.21.15.7:53940) 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.792+0000 7f80d5df0640 1 -- 172.21.15.7:0/1116104712 learned_addr learned my addr 172.21.15.7:0/1116104712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.792+0000 7f80d55ef640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f80d015e5b0 0x7f80d0162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39926/0 (socket says 172.21.15.7:39926) 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.793+0000 7f80d4dee640 1 --2- 172.21.15.7:0/1116104712 >> v2:172.21.15.18:3300/0 conn(0x7f80d00d24e0 0x7f80d0162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.793+0000 7f80d5df0640 1 -- 172.21.15.7:0/1116104712 >> v2:172.21.15.18:3300/0 conn(0x7f80d00d24e0 msgr2=0x7f80d0162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.793+0000 7f80d5df0640 1 --2- 172.21.15.7:0/1116104712 >> v2:172.21.15.18:3300/0 conn(0x7f80d00d24e0 0x7f80d0162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.793+0000 7f80d5df0640 1 -- 172.21.15.7:0/1116104712 >> v1:172.21.15.7:6789/0 conn(0x7f80d015e5b0 legacy=0x7f80d0162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.793+0000 7f80c3fff640 1 -- 172.21.15.7:0/1116104712 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3638304612 0 0) 0x7f80d00d2980 con 0x7f80d015e5b0 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.793+0000 7f80d5df0640 1 -- 172.21.15.7:0/1116104712 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d00d3200 con 0x7f80d015e990 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80d5df0640 1 --2- 172.21.15.7:0/1116104712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e990 0x7f80d00cf420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009e50 tx=0x7f80c802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91fa451343dff24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80c3fff640 1 -- 172.21.15.7:0/1116104712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8036070 con 0x7f80d015e990 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80c3fff640 1 -- 172.21.15.7:0/1116104712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c8031600 con 0x7f80d015e990 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80c3fff640 1 -- 172.21.15.7:0/1116104712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c80319d0 con 0x7f80d015e990 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80d7853640 1 -- 172.21.15.7:0/1116104712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e990 msgr2=0x7f80d00cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80d7853640 1 --2- 172.21.15.7:0/1116104712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e990 0x7f80d00cf420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009e50 tx=0x7f80c802a760 comp rx=0 tx=0).stop 2024-09-15T23:37:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80d7853640 1 -- 172.21.15.7:0/1116104712 shutdown_connections 2024-09-15T23:37:36.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80d7853640 1 --2- 172.21.15.7:0/1116104712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e990 0x7f80d00cf420 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80d7853640 1 --2- 172.21.15.7:0/1116104712 >> v2:172.21.15.18:3300/0 conn(0x7f80d00d24e0 0x7f80d0162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.794+0000 7f80d7853640 1 -- 172.21.15.7:0/1116104712 >> 172.21.15.7:0/1116104712 conn(0x7f80d008bfd0 msgr2=0x7f80d00d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:36.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.795+0000 7f80d7853640 1 -- 172.21.15.7:0/1116104712 shutdown_connections 2024-09-15T23:37:36.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.795+0000 7f80d7853640 1 -- 172.21.15.7:0/1116104712 wait complete. 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.796+0000 7f80d7853640 1 Processor -- start 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.796+0000 7f80d7853640 1 -- start start 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.796+0000 7f80d7853640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d00d24e0 0x7f80d014cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.796+0000 7f80d7853640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e5b0 0x7f80d00ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.796+0000 7f80d7853640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80d015e990 0x7f80d015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.796+0000 7f80d7853640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80d00b3b40 con 0x7f80d015e5b0 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.796+0000 7f80d7853640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80d00b39c0 con 0x7f80d00d24e0 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.796+0000 7f80d7853640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f80d0162fa0 con 0x7f80d015e990 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d4dee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e5b0 0x7f80d00ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d4dee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e5b0 0x7f80d00ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53956/0 (socket says 172.21.15.7:53956) 2024-09-15T23:37:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d4dee640 1 -- 172.21.15.7:0/1604755225 learned_addr learned my addr 172.21.15.7:0/1604755225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d55ef640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d00d24e0 0x7f80d014cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d5df0640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80d015e990 0x7f80d015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d4dee640 1 -- 172.21.15.7:0/1604755225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80d015e990 msgr2=0x7f80d015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d4dee640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80d015e990 0x7f80d015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d4dee640 1 -- 172.21.15.7:0/1604755225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d00d24e0 msgr2=0x7f80d014cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d4dee640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d00d24e0 0x7f80d014cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d4dee640 1 -- 172.21.15.7:0/1604755225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d00cbb40 con 0x7f80d015e5b0 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d4dee640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e5b0 0x7f80d00ccbe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f80c400e930 tx=0x7f80c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80c1ffb640 1 -- 172.21.15.7:0/1604755225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4017070 con 0x7f80d015e5b0 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80c1ffb640 1 -- 172.21.15.7:0/1604755225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c400ca00 con 0x7f80d015e5b0 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80d7853640 1 -- 172.21.15.7:0/1604755225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d00cbe20 con 0x7f80d015e5b0 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.797+0000 7f80c1ffb640 1 -- 172.21.15.7:0/1604755225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c400cdb0 con 0x7f80d015e5b0 2024-09-15T23:37:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.798+0000 7f80d7853640 1 -- 172.21.15.7:0/1604755225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d00cc3e0 con 0x7f80d015e5b0 2024-09-15T23:37:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.799+0000 7f80d7853640 1 -- 172.21.15.7:0/1604755225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d0068f20 con 0x7f80d015e5b0 2024-09-15T23:37:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.799+0000 7f80c1ffb640 1 -- 172.21.15.7:0/1604755225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 90949+0+0 (secure 0 0 0) 0x7f80c401a070 con 0x7f80d015e5b0 2024-09-15T23:37:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.800+0000 7f80c1ffb640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f80b406c400 0x7f80b406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.800+0000 7f80c1ffb640 1 -- 172.21.15.7:0/1604755225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6445+0+0 (secure 0 0 0) 0x7f80c408bb20 con 0x7f80d015e5b0 2024-09-15T23:37:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.800+0000 7f80d55ef640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f80b406c400 0x7f80b406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.800+0000 7f80d55ef640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f80b406c400 0x7f80b406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello connection peer type does not match what peer advertises 16 != 2 2024-09-15T23:37:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.800+0000 7f80d55ef640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f80b406c400 0x7f80b406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.800+0000 7f80c1ffb640 1 -- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f80b406c400 msgr2=0x7f80b406e8c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T23:37:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.800+0000 7f80c1ffb640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f80b406c400 0x7f80b406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:36.804+0000 7f80c1ffb640 1 -- 172.21.15.7:0/1604755225 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f80c40599a0 con 0x7f80d015e5b0 2024-09-15T23:37:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.016+0000 7f80d7853640 1 -- 172.21.15.7:0/1604755225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f80d00d2980 con 0x7f80d015e5b0 2024-09-15T23:37:37.242 INFO:tasks.ceph.mgr.x.smithi018.stderr:2024-09-15T23:37:37.241+0000 7f96a07d7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:37:37.315 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.314+0000 7f7b80e90040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:37:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.321+0000 7f80c1ffb640 1 -- 172.21.15.7:0/1604755225 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 12) ==== 90557+0+0 (secure 0 0 0) 0x7f80c4056010 con 0x7f80d015e5b0 2024-09-15T23:37:37.371 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b02654640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.371 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b02654640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.371 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b02654640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.371 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b02654640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.372 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b02654640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.372 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b0565a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.372 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b0565a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.372 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b0565a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.372 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b0565a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.372 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-15T23:37:37.370+0000 7f7b0565a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:37:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.682+0000 7f80c1ffb640 1 -- 172.21.15.7:0/1604755225 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f80c4078700 con 0x7f80d015e5b0 2024-09-15T23:37:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 -- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e5b0 msgr2=0x7f80d00ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e5b0 0x7f80d00ccbe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f80c400e930 tx=0x7f80c400edf0 comp rx=0 tx=0).stop 2024-09-15T23:37:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 -- 172.21.15.7:0/1604755225 shutdown_connections 2024-09-15T23:37:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80d015e990 0x7f80d015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d015e5b0 0x7f80d00ccbe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.7:6832/4033779377,v1:172.21.15.7:6833/4033779377] conn(0x7f80b406c400 0x7f80b406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 --2- 172.21.15.7:0/1604755225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d00d24e0 0x7f80d014cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 -- 172.21.15.7:0/1604755225 >> 172.21.15.7:0/1604755225 conn(0x7f80d008bfd0 msgr2=0x7f80d00d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 -- 172.21.15.7:0/1604755225 shutdown_connections 2024-09-15T23:37:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.685+0000 7f80d7853640 1 -- 172.21.15.7:0/1604755225 wait complete. 2024-09-15T23:37:37.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-15T23:37:37.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.847+0000 7feae519f640 1 Processor -- start 2024-09-15T23:37:37.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.847+0000 7feae519f640 1 -- start start 2024-09-15T23:37:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.848+0000 7feae519f640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feae00d0f80 0x7feae00cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.848+0000 7feae519f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae0162300 con 0x7feae00d0af0 2024-09-15T23:37:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.848+0000 7feae519f640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae01625e0 con 0x7feae015e760 2024-09-15T23:37:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.848+0000 7feae519f640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7feae0162f10 con 0x7feae00d0f80 2024-09-15T23:37:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.848+0000 7feadf59e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feae00d0f80 0x7feae00cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.848+0000 7feadf59e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feae00d0f80 0x7feae00cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38426/0 (socket says 172.21.15.7:38426) 2024-09-15T23:37:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.848+0000 7feadf59e640 1 -- 172.21.15.7:0/4165919171 learned_addr learned my addr 172.21.15.7:0/4165919171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.848+0000 7feade59c640 1 --1- 172.21.15.7:0/4165919171 >> v1:172.21.15.7:6789/0 conn(0x7feae00d0af0 0x7feae00cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57608/0 (socket says 172.21.15.7:57608) 2024-09-15T23:37:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.849+0000 7feadf59e640 1 -- 172.21.15.7:0/4165919171 >> v1:172.21.15.18:6789/0 conn(0x7feae015e760 legacy=0x7feae00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.849+0000 7feadf59e640 1 -- 172.21.15.7:0/4165919171 >> v1:172.21.15.7:6789/0 conn(0x7feae00d0af0 legacy=0x7feae00cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.849+0000 7feadf59e640 1 -- 172.21.15.7:0/4165919171 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae01628c0 con 0x7feae00d0f80 2024-09-15T23:37:37.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.849+0000 7feadf59e640 1 --2- 172.21.15.7:0/4165919171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0f80 0x7feae00cdbf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fead8009870 tx=0x7fead802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffaf1e828554fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:37.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.849+0000 7feaddd9b640 1 -- 172.21.15.7:0/4165919171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead8037070 con 0x7feae00d0f80 2024-09-15T23:37:37.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.850+0000 7feaddd9b640 1 -- 172.21.15.7:0/4165919171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead802ae50 con 0x7feae00d0f80 2024-09-15T23:37:37.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.850+0000 7feaddd9b640 1 -- 172.21.15.7:0/4165919171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead8032860 con 0x7feae00d0f80 2024-09-15T23:37:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.850+0000 7feae519f640 1 -- 172.21.15.7:0/4165919171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0f80 msgr2=0x7feae00cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.850+0000 7feae519f640 1 --2- 172.21.15.7:0/4165919171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0f80 0x7feae00cdbf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fead8009870 tx=0x7fead802a300 comp rx=0 tx=0).stop 2024-09-15T23:37:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.850+0000 7feae519f640 1 -- 172.21.15.7:0/4165919171 shutdown_connections 2024-09-15T23:37:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.850+0000 7feae519f640 1 --2- 172.21.15.7:0/4165919171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0f80 0x7feae00cdbf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.850+0000 7feae519f640 1 -- 172.21.15.7:0/4165919171 >> 172.21.15.7:0/4165919171 conn(0x7feae00d2430 msgr2=0x7feae00d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.850+0000 7feae519f640 1 -- 172.21.15.7:0/4165919171 shutdown_connections 2024-09-15T23:37:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.850+0000 7feae519f640 1 -- 172.21.15.7:0/4165919171 wait complete. 2024-09-15T23:37:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feae519f640 1 Processor -- start 2024-09-15T23:37:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feae519f640 1 -- start start 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feae519f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0af0 0x7feae00b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feae519f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feae00d0f80 0x7feae0153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feae519f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feae015e760 0x7feae00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feae519f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feae00d0710 con 0x7feae00d0af0 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feae519f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feae00d1390 con 0x7feae00d0f80 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feae519f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7feae00d0890 con 0x7feae015e760 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feaded9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0af0 0x7feae00b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feaded9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0af0 0x7feae00b7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38438/0 (socket says 172.21.15.7:38438) 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.852+0000 7feaded9d640 1 -- 172.21.15.7:0/2523518260 learned_addr learned my addr 172.21.15.7:0/2523518260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feade59c640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feae00d0f80 0x7feae0153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feadf59e640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feae015e760 0x7feae00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feaded9d640 1 -- 172.21.15.7:0/2523518260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feae015e760 msgr2=0x7feae00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feaded9d640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feae015e760 0x7feae00ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feaded9d640 1 -- 172.21.15.7:0/2523518260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feae00d0f80 msgr2=0x7feae0153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feaded9d640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feae00d0f80 0x7feae0153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feaded9d640 1 -- 172.21.15.7:0/2523518260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae00b5390 con 0x7feae00d0af0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feaded9d640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0af0 0x7feae00b7b40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fead000c790 tx=0x7fead000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feac77fe640 1 -- 172.21.15.7:0/2523518260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0016020 con 0x7feae00d0af0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feac77fe640 1 -- 172.21.15.7:0/2523518260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead0006d50 con 0x7feae00d0af0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae00b5670 con 0x7feae00d0af0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feac77fe640 1 -- 172.21.15.7:0/2523518260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0003bb0 con 0x7feae00d0af0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.853+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae00b5c30 con 0x7feae00d0af0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.854+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae00d0000 con 0x7feae00d0af0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.855+0000 7feac77fe640 1 -- 172.21.15.7:0/2523518260 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 90557+0+0 (secure 0 0 0) 0x7fead000a850 con 0x7feae00d0af0 2024-09-15T23:37:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.855+0000 7feac77fe640 1 -- 172.21.15.7:0/2523518260 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6789+0+0 (secure 0 0 0) 0x7fead008ae80 con 0x7feae00d0af0 2024-09-15T23:37:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:37.860+0000 7feac77fe640 1 -- 172.21.15.7:0/2523518260 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fead0020020 con 0x7feae00d0af0 2024-09-15T23:37:38.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.034+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7feae00d32d0 con 0x7feae00d0af0 2024-09-15T23:37:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.327+0000 7feac77fe640 1 -- 172.21.15.7:0/2523518260 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 13) ==== 90684+0+0 (secure 0 0 0) 0x7fead0054d00 con 0x7feae00d0af0 2024-09-15T23:37:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.327+0000 7feac77fe640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7feacc0720c0 0x7feacc0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.329+0000 7feade59c640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7feacc0720c0 0x7feacc0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.331+0000 7feade59c640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7feacc0720c0 0x7feacc0744b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feae00c8b00 tx=0x7fead4000ed0 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.686+0000 7feac77fe640 1 -- 172.21.15.7:0/2523518260 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7fead0078ac0 con 0x7feae00d0af0 2024-09-15T23:37:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7feacc0720c0 msgr2=0x7feacc0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7feacc0720c0 0x7feacc0744b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feae00c8b00 tx=0x7fead4000ed0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0af0 msgr2=0x7feae00b7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0af0 0x7feae00b7b40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fead000c790 tx=0x7fead000cc50 comp rx=0 tx=0).stop 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 shutdown_connections 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7feacc0720c0 0x7feacc0744b0 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feae00c8b00 tx=0x7fead4000ed0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feae015e760 0x7feae00ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feae00d0f80 0x7feae0153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 --2- 172.21.15.7:0/2523518260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae00d0af0 0x7feae00b7b40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.691+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 >> 172.21.15.7:0/2523518260 conn(0x7feae00d2430 msgr2=0x7feae0071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.692+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 shutdown_connections 2024-09-15T23:37:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.692+0000 7feae519f640 1 -- 172.21.15.7:0/2523518260 wait complete. 2024-09-15T23:37:38.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-15T23:37:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.819+0000 7ff79465e640 1 Processor -- start 2024-09-15T23:37:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.819+0000 7ff79465e640 1 -- start start 2024-09-15T23:37:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.820+0000 7ff79465e640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff78c15fa70 0x7ff78c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.820+0000 7ff79465e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c09b150 0x7ff78c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.820+0000 7ff79465e640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff78c0701d0 con 0x7ff78c15fe50 2024-09-15T23:37:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.820+0000 7ff79465e640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff78c0716c0 con 0x7ff78c15fa70 2024-09-15T23:37:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.820+0000 7ff79465e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff78c09ba40 con 0x7ff78c09b150 2024-09-15T23:37:38.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.820+0000 7ff792bfb640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c09b150 0x7ff78c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.820+0000 7ff7923fa640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff78c15fa70 0x7ff78c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.821+0000 7ff791bf9640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7ff78c15fe50 0x7ff78c09aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:43718/0 (socket says 172.21.15.7:43718) 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.821+0000 7ff791bf9640 1 -- 172.21.15.7:0/973941402 learned_addr learned my addr 172.21.15.7:0/973941402 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.821+0000 7ff7923fa640 1 -- 172.21.15.7:0/973941402 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c09b150 msgr2=0x7ff78c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.821+0000 7ff7923fa640 1 --2- 172.21.15.7:0/973941402 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c09b150 0x7ff78c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.821+0000 7ff7923fa640 1 -- 172.21.15.7:0/973941402 >> v1:172.21.15.18:6789/0 conn(0x7ff78c15fe50 legacy=0x7ff78c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.821+0000 7ff7923fa640 1 -- 172.21.15.7:0/973941402 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c09bd20 con 0x7ff78c15fa70 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.821+0000 7ff792bfb640 1 --2- 172.21.15.7:0/973941402 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c09b150 0x7ff78c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.822+0000 7ff7923fa640 1 --2- 172.21.15.7:0/973941402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff78c15fa70 0x7ff78c0cce50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff7880097a0 tx=0x7ff78802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c72eb6df7b1897e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.822+0000 7ff7913f8640 1 -- 172.21.15.7:0/973941402 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788037070 con 0x7ff78c15fa70 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.822+0000 7ff7913f8640 1 -- 172.21.15.7:0/973941402 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff788032440 con 0x7ff78c15fa70 2024-09-15T23:37:38.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.822+0000 7ff7913f8640 1 -- 172.21.15.7:0/973941402 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788032810 con 0x7ff78c15fa70 2024-09-15T23:37:38.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.822+0000 7ff79465e640 1 -- 172.21.15.7:0/973941402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff78c15fa70 msgr2=0x7ff78c0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:38.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.822+0000 7ff79465e640 1 --2- 172.21.15.7:0/973941402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff78c15fa70 0x7ff78c0cce50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff7880097a0 tx=0x7ff78802a180 comp rx=0 tx=0).stop 2024-09-15T23:37:38.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.823+0000 7ff79465e640 1 -- 172.21.15.7:0/973941402 shutdown_connections 2024-09-15T23:37:38.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.823+0000 7ff79465e640 1 --2- 172.21.15.7:0/973941402 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c09b150 0x7ff78c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.823+0000 7ff79465e640 1 --2- 172.21.15.7:0/973941402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff78c15fa70 0x7ff78c0cce50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.823+0000 7ff79465e640 1 -- 172.21.15.7:0/973941402 >> 172.21.15.7:0/973941402 conn(0x7ff78c0d2420 msgr2=0x7ff78c0d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:38.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.823+0000 7ff79465e640 1 -- 172.21.15.7:0/973941402 shutdown_connections 2024-09-15T23:37:38.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.823+0000 7ff79465e640 1 -- 172.21.15.7:0/973941402 wait complete. 2024-09-15T23:37:38.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.824+0000 7ff79465e640 1 Processor -- start 2024-09-15T23:37:38.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.824+0000 7ff79465e640 1 -- start start 2024-09-15T23:37:38.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.824+0000 7ff79465e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c09b150 0x7ff78c0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:38.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.824+0000 7ff79465e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c15fa70 0x7ff78c1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:38.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff79465e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff78c15fe50 0x7ff78c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:38.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff79465e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c0d31e0 con 0x7ff78c09b150 2024-09-15T23:37:38.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff79465e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c0d3060 con 0x7ff78c15fe50 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff79465e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff78c0d3360 con 0x7ff78c15fa70 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff7923fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c09b150 0x7ff78c0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff7923fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c09b150 0x7ff78c0b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38452/0 (socket says 172.21.15.7:38452) 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff7923fa640 1 -- 172.21.15.7:0/3415522097 learned_addr learned my addr 172.21.15.7:0/3415522097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff791bf9640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c15fa70 0x7ff78c1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff792bfb640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff78c15fe50 0x7ff78c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff7923fa640 1 -- 172.21.15.7:0/3415522097 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c15fa70 msgr2=0x7ff78c1f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff7923fa640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c15fa70 0x7ff78c1f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff7923fa640 1 -- 172.21.15.7:0/3415522097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff78c15fe50 msgr2=0x7ff78c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff7923fa640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff78c15fe50 0x7ff78c0b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff7923fa640 1 -- 172.21.15.7:0/3415522097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c0b5310 con 0x7ff78c09b150 2024-09-15T23:37:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff7923fa640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c09b150 0x7ff78c0b6060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff788035040 tx=0x7ff7880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff782ffd640 1 -- 172.21.15.7:0/3415522097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788037040 con 0x7ff78c09b150 2024-09-15T23:37:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff782ffd640 1 -- 172.21.15.7:0/3415522097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff788030500 con 0x7ff78c09b150 2024-09-15T23:37:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c0b7e30 con 0x7ff78c09b150 2024-09-15T23:37:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff782ffd640 1 -- 172.21.15.7:0/3415522097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788030910 con 0x7ff78c09b150 2024-09-15T23:37:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.825+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c0b5860 con 0x7ff78c09b150 2024-09-15T23:37:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.826+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78c071f80 con 0x7ff78c09b150 2024-09-15T23:37:38.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.827+0000 7ff782ffd640 1 -- 172.21.15.7:0/3415522097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 90684+0+0 (secure 0 0 0) 0x7ff788044020 con 0x7ff78c09b150 2024-09-15T23:37:38.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.827+0000 7ff782ffd640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff77006c100 0x7ff77006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:38.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.827+0000 7ff782ffd640 1 -- 172.21.15.7:0/3415522097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6351+0+0 (secure 0 0 0) 0x7ff7880ab630 con 0x7ff78c09b150 2024-09-15T23:37:38.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.828+0000 7ff791bf9640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff77006c100 0x7ff77006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:38.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.828+0000 7ff791bf9640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff77006c100 0x7ff77006e5c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0b19d0 tx=0x7ff77c009040 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:38.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:38.835+0000 7ff782ffd640 1 -- 172.21.15.7:0/3415522097 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff788079de0 con 0x7ff78c09b150 2024-09-15T23:37:39.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.009+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff78c0b3d50 con 0x7ff78c09b150 2024-09-15T23:37:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.693+0000 7ff782ffd640 1 -- 172.21.15.7:0/3415522097 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 14) ==== 90790+0+0 (secure 0 0 0) 0x7ff788075d50 con 0x7ff78c09b150 2024-09-15T23:37:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.696+0000 7ff782ffd640 1 -- 172.21.15.7:0/3415522097 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7ff7880934e0 con 0x7ff78c09b150 2024-09-15T23:37:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-15T23:37:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff77006c100 msgr2=0x7ff77006e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff77006c100 0x7ff77006e5c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0b19d0 tx=0x7ff77c009040 comp rx=0 tx=0).stop 2024-09-15T23:37:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c09b150 msgr2=0x7ff78c0b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c09b150 0x7ff78c0b6060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff788035040 tx=0x7ff7880046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 shutdown_connections 2024-09-15T23:37:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff77006c100 0x7ff77006e5c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff78c15fe50 0x7ff78c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff78c15fa70 0x7ff78c1f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 --2- 172.21.15.7:0/3415522097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c09b150 0x7ff78c0b6060 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 >> 172.21.15.7:0/3415522097 conn(0x7ff78c0d2420 msgr2=0x7ff78c09aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:39.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.700+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 shutdown_connections 2024-09-15T23:37:39.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.701+0000 7ff79465e640 1 -- 172.21.15.7:0/3415522097 wait complete. 2024-09-15T23:37:39.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2024-09-15T23:37:39.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.821+0000 7f3b8feaf640 1 Processor -- start 2024-09-15T23:37:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.821+0000 7f3b8feaf640 1 -- start start 2024-09-15T23:37:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8feaf640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3b8815e870 0x7f3b8809b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8feaf640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b880d0c50 0x7f3b880ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8feaf640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b880d14a0 con 0x7f3b880ce590 2024-09-15T23:37:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8feaf640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3b8815ec50 con 0x7f3b880d0c50 2024-09-15T23:37:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8feaf640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3b880d1780 con 0x7f3b8815e870 2024-09-15T23:37:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8d44a640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b880d0c50 0x7f3b880ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8e44c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3b880ce590 0x7f3b8806ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57610/0 (socket says 172.21.15.7:57610) 2024-09-15T23:37:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8e44c640 1 -- 172.21.15.7:0/3511233598 learned_addr learned my addr 172.21.15.7:0/3511233598 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8d44a640 1 -- 172.21.15.7:0/3511233598 >> v2:172.21.15.18:3300/0 conn(0x7f3b8815e870 msgr2=0x7f3b8809b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8dc4b640 1 --2- 172.21.15.7:0/3511233598 >> v2:172.21.15.18:3300/0 conn(0x7f3b8815e870 0x7f3b8809b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8d44a640 1 --2- 172.21.15.7:0/3511233598 >> v2:172.21.15.18:3300/0 conn(0x7f3b8815e870 0x7f3b8809b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8d44a640 1 -- 172.21.15.7:0/3511233598 >> v1:172.21.15.7:6789/0 conn(0x7f3b880ce590 legacy=0x7f3b8806ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8d44a640 1 -- 172.21.15.7:0/3511233598 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b880cd850 con 0x7f3b880d0c50 2024-09-15T23:37:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.822+0000 7f3b8cc49640 1 -- 172.21.15.7:0/3511233598 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1302136087 0 0) 0x7f3b880d14a0 con 0x7f3b880ce590 2024-09-15T23:37:39.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8d44a640 1 --2- 172.21.15.7:0/3511233598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 0x7f3b880ce050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b880733a0 tx=0x7f3b7c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e849d6b56e3ab15f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:39.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8cc49640 1 -- 172.21.15.7:0/3511233598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c037070 con 0x7f3b880d0c50 2024-09-15T23:37:39.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8cc49640 1 -- 172.21.15.7:0/3511233598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c032440 con 0x7f3b880d0c50 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8cc49640 1 -- 172.21.15.7:0/3511233598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c032810 con 0x7f3b880d0c50 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8feaf640 1 -- 172.21.15.7:0/3511233598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 msgr2=0x7f3b880ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8feaf640 1 --2- 172.21.15.7:0/3511233598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 0x7f3b880ce050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b880733a0 tx=0x7f3b7c02a230 comp rx=0 tx=0).stop 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8feaf640 1 -- 172.21.15.7:0/3511233598 shutdown_connections 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8feaf640 1 --2- 172.21.15.7:0/3511233598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 0x7f3b880ce050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8feaf640 1 --2- 172.21.15.7:0/3511233598 >> v2:172.21.15.18:3300/0 conn(0x7f3b8815e870 0x7f3b8809b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.823+0000 7f3b8feaf640 1 -- 172.21.15.7:0/3511233598 >> 172.21.15.7:0/3511233598 conn(0x7f3b880b3f30 msgr2=0x7f3b880b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.824+0000 7f3b8feaf640 1 -- 172.21.15.7:0/3511233598 shutdown_connections 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.824+0000 7f3b8feaf640 1 -- 172.21.15.7:0/3511233598 wait complete. 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.825+0000 7f3b8feaf640 1 Processor -- start 2024-09-15T23:37:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.825+0000 7f3b8feaf640 1 -- start start 2024-09-15T23:37:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.825+0000 7f3b8feaf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b880ce590 0x7f3b880c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.825+0000 7f3b8feaf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 0x7f3b8815d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.825+0000 7f3b8feaf640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8815e870 0x7f3b8815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.825+0000 7f3b8feaf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880d10a0 con 0x7f3b880d0c50 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8feaf640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880cd550 con 0x7f3b880ce590 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8feaf640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f3b880d1220 con 0x7f3b8815e870 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8d44a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 0x7f3b8815d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8d44a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 0x7f3b8815d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38472/0 (socket says 172.21.15.7:38472) 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8d44a640 1 -- 172.21.15.7:0/62557533 learned_addr learned my addr 172.21.15.7:0/62557533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8d44a640 1 -- 172.21.15.7:0/62557533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8815e870 msgr2=0x7f3b8815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8d44a640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8815e870 0x7f3b8815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8d44a640 1 -- 172.21.15.7:0/62557533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b880ce590 msgr2=0x7f3b880c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8dc4b640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b880ce590 0x7f3b880c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8d44a640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b880ce590 0x7f3b880c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8d44a640 1 -- 172.21.15.7:0/62557533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b880cbaa0 con 0x7f3b880d0c50 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8d44a640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 0x7f3b8815d1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009890 tx=0x7f3b7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b867fc640 1 -- 172.21.15.7:0/62557533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c041070 con 0x7f3b880d0c50 2024-09-15T23:37:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b867fc640 1 -- 172.21.15.7:0/62557533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c03c950 con 0x7f3b880d0c50 2024-09-15T23:37:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b880cbd80 con 0x7f3b880d0c50 2024-09-15T23:37:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b867fc640 1 -- 172.21.15.7:0/62557533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c037040 con 0x7f3b880d0c50 2024-09-15T23:37:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.826+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b880cc340 con 0x7f3b880d0c50 2024-09-15T23:37:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.827+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b88069140 con 0x7f3b880d0c50 2024-09-15T23:37:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.828+0000 7f3b867fc640 1 -- 172.21.15.7:0/62557533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 14) ==== 90790+0+0 (secure 0 0 0) 0x7f3b7c044070 con 0x7f3b880d0c50 2024-09-15T23:37:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.828+0000 7f3b867fc640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f3b7406c220 0x7f3b7406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.828+0000 7f3b867fc640 1 -- 172.21.15.7:0/62557533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f3b7c0ac1d0 con 0x7f3b880d0c50 2024-09-15T23:37:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.828+0000 7f3b8dc4b640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f3b7406c220 0x7f3b7406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.829+0000 7f3b8dc4b640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f3b7406c220 0x7f3b7406e6e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3b78008e50 tx=0x7f3b780083a0 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:39.833+0000 7f3b867fc640 1 -- 172.21.15.7:0/62557533 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3b88069140 con 0x7f3b880d0c50 2024-09-15T23:37:40.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.018+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) -- 0x7f3b880d14a0 con 0x7f3b880d0c50 2024-09-15T23:37:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.021+0000 7f3b867fc640 1 -- 172.21.15.7:0/62557533 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v13) ==== 115+0+0 (secure 0 0 0) 0x7f3b7c07aae0 con 0x7f3b880d0c50 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f3b7406c220 msgr2=0x7f3b7406e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f3b7406c220 0x7f3b7406e6e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3b78008e50 tx=0x7f3b780083a0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 msgr2=0x7f3b8815d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 0x7f3b8815d1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009890 tx=0x7f3b7c0046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 shutdown_connections 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f3b7406c220 0x7f3b7406e6e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8815e870 0x7f3b8815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b880d0c50 0x7f3b8815d1a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 --2- 172.21.15.7:0/62557533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b880ce590 0x7f3b880c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 >> 172.21.15.7:0/62557533 conn(0x7f3b880b3f30 msgr2=0x7f3b8806e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 shutdown_connections 2024-09-15T23:37:40.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.024+0000 7f3b8feaf640 1 -- 172.21.15.7:0/62557533 wait complete. 2024-09-15T23:37:40.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-15T23:37:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.139+0000 7f40537c4640 1 Processor -- start 2024-09-15T23:37:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.140+0000 7f40537c4640 1 -- start start 2024-09-15T23:37:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.140+0000 7f40537c4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e5b0 0x7f404c162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.140+0000 7f40537c4640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f404c15e990 0x7f404c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.140+0000 7f40537c4640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f404c0d2980 con 0x7f404c0d24e0 2024-09-15T23:37:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.140+0000 7f40537c4640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f404c0d2c40 con 0x7f404c15e990 2024-09-15T23:37:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.140+0000 7f40537c4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f404c0d2f20 con 0x7f404c15e5b0 2024-09-15T23:37:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.141+0000 7f4051560640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e5b0 0x7f404c162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.141+0000 7f4051d61640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f404c15e990 0x7f404c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.141+0000 7f4050d5f640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f404c0d24e0 0x7f404c162890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:43728/0 (socket says 172.21.15.7:43728) 2024-09-15T23:37:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.141+0000 7f4050d5f640 1 -- 172.21.15.7:0/1088832334 learned_addr learned my addr 172.21.15.7:0/1088832334 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.141+0000 7f4051d61640 1 -- 172.21.15.7:0/1088832334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e5b0 msgr2=0x7f404c162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.141+0000 7f4051d61640 1 --2- 172.21.15.7:0/1088832334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e5b0 0x7f404c162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.141+0000 7f4051d61640 1 -- 172.21.15.7:0/1088832334 >> v1:172.21.15.18:6789/0 conn(0x7f404c0d24e0 legacy=0x7f404c162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.141+0000 7f4051d61640 1 -- 172.21.15.7:0/1088832334 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c0d3200 con 0x7f404c15e990 2024-09-15T23:37:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.141+0000 7f4051560640 1 --2- 172.21.15.7:0/1088832334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e5b0 0x7f404c162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.142+0000 7f4051d61640 1 --2- 172.21.15.7:0/1088832334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f404c15e990 0x7f404c0cf420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4044009d90 tx=0x7f404402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17758aa36dec177a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.142+0000 7f4043fff640 1 -- 172.21.15.7:0/1088832334 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044007740 con 0x7f404c15e990 2024-09-15T23:37:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.142+0000 7f4043fff640 1 -- 172.21.15.7:0/1088832334 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40440078e0 con 0x7f404c15e990 2024-09-15T23:37:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.142+0000 7f4043fff640 1 -- 172.21.15.7:0/1088832334 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044007cb0 con 0x7f404c15e990 2024-09-15T23:37:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.143+0000 7f40537c4640 1 -- 172.21.15.7:0/1088832334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f404c15e990 msgr2=0x7f404c0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.143+0000 7f40537c4640 1 --2- 172.21.15.7:0/1088832334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f404c15e990 0x7f404c0cf420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4044009d90 tx=0x7f404402a660 comp rx=0 tx=0).stop 2024-09-15T23:37:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.143+0000 7f40537c4640 1 -- 172.21.15.7:0/1088832334 shutdown_connections 2024-09-15T23:37:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.143+0000 7f40537c4640 1 --2- 172.21.15.7:0/1088832334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f404c15e990 0x7f404c0cf420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.143+0000 7f40537c4640 1 --2- 172.21.15.7:0/1088832334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e5b0 0x7f404c162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.143+0000 7f40537c4640 1 -- 172.21.15.7:0/1088832334 >> 172.21.15.7:0/1088832334 conn(0x7f404c08bfd0 msgr2=0x7f404c0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.143+0000 7f40537c4640 1 -- 172.21.15.7:0/1088832334 shutdown_connections 2024-09-15T23:37:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.143+0000 7f40537c4640 1 -- 172.21.15.7:0/1088832334 wait complete. 2024-09-15T23:37:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.144+0000 7f40537c4640 1 Processor -- start 2024-09-15T23:37:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.144+0000 7f40537c4640 1 -- start start 2024-09-15T23:37:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f40537c4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f404c0d24e0 0x7f404c0c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f40537c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404c15e5b0 0x7f404c14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f40537c4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e990 0x7f404c159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f40537c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f404c0b3b40 con 0x7f404c15e5b0 2024-09-15T23:37:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f40537c4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f404c0b39c0 con 0x7f404c0d24e0 2024-09-15T23:37:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f40537c4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f404c162fa0 con 0x7f404c15e990 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4050d5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404c15e5b0 0x7f404c14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4050d5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404c15e5b0 0x7f404c14da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38488/0 (socket says 172.21.15.7:38488) 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4050d5f640 1 -- 172.21.15.7:0/3855418630 learned_addr learned my addr 172.21.15.7:0/3855418630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4051d61640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e990 0x7f404c159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4051560640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f404c0d24e0 0x7f404c0c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4050d5f640 1 -- 172.21.15.7:0/3855418630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e990 msgr2=0x7f404c159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4050d5f640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e990 0x7f404c159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4050d5f640 1 -- 172.21.15.7:0/3855418630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f404c0d24e0 msgr2=0x7f404c0c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4050d5f640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f404c0d24e0 0x7f404c0c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.145+0000 7f4050d5f640 1 -- 172.21.15.7:0/3855418630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c156d20 con 0x7f404c15e5b0 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.146+0000 7f4050d5f640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404c15e5b0 0x7f404c14da10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f403c00c430 tx=0x7f403c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.146+0000 7f4041ffb640 1 -- 172.21.15.7:0/3855418630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c017070 con 0x7f404c15e5b0 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.146+0000 7f4041ffb640 1 -- 172.21.15.7:0/3855418630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403c005d30 con 0x7f404c15e5b0 2024-09-15T23:37:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.146+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c0c8fe0 con 0x7f404c15e5b0 2024-09-15T23:37:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.146+0000 7f4041ffb640 1 -- 172.21.15.7:0/3855418630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0072f0 con 0x7f404c15e5b0 2024-09-15T23:37:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.146+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c0c8740 con 0x7f404c15e5b0 2024-09-15T23:37:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.147+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c162350 con 0x7f404c15e5b0 2024-09-15T23:37:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.148+0000 7f4041ffb640 1 -- 172.21.15.7:0/3855418630 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 14) ==== 90790+0+0 (secure 0 0 0) 0x7f403c01a070 con 0x7f404c15e5b0 2024-09-15T23:37:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.148+0000 7f4041ffb640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f403006c220 0x7f403006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.148+0000 7f4041ffb640 1 -- 172.21.15.7:0/3855418630 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f403c08a730 con 0x7f404c15e5b0 2024-09-15T23:37:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.148+0000 7f4051560640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f403006c220 0x7f403006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.149+0000 7f4051560640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f403006c220 0x7f403006e6e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4048009560 tx=0x7f4048009190 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.152+0000 7f4041ffb640 1 -- 172.21.15.7:0/3855418630 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f403c058850 con 0x7f404c15e5b0 2024-09-15T23:37:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.325+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f404c0ba560 con 0x7f404c15e5b0 2024-09-15T23:37:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.328+0000 7f4041ffb640 1 -- 172.21.15.7:0/3855418630 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) ==== 104+0+0 (secure 0 0 0) 0x7f403c05cb50 con 0x7f404c15e5b0 2024-09-15T23:37:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f403006c220 msgr2=0x7f403006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f403006c220 0x7f403006e6e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4048009560 tx=0x7f4048009190 comp rx=0 tx=0).stop 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404c15e5b0 msgr2=0x7f404c14da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404c15e5b0 0x7f404c14da10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f403c00c430 tx=0x7f403c00c8f0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 shutdown_connections 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f403006c220 0x7f403006e6e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f404c15e990 0x7f404c159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404c15e5b0 0x7f404c14da10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 --2- 172.21.15.7:0/3855418630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f404c0d24e0 0x7f404c0c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.330+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 >> 172.21.15.7:0/3855418630 conn(0x7f404c08bfd0 msgr2=0x7f404c0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.331+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 shutdown_connections 2024-09-15T23:37:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.331+0000 7f40537c4640 1 -- 172.21.15.7:0/3855418630 wait complete. 2024-09-15T23:37:40.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-09-15T23:37:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.445+0000 7fdf720d0640 1 Processor -- start 2024-09-15T23:37:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.445+0000 7fdf720d0640 1 -- start start 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.446+0000 7fdf720d0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 0x7fdf6c0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.446+0000 7fdf720d0640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdf6c0b5570 0x7fdf6c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.446+0000 7fdf720d0640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf6c0b5c40 0x7fdf6c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.446+0000 7fdf720d0640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdf6c0d5540 con 0x7fdf6c0b5c40 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.446+0000 7fdf720d0640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fdf6c0d4d30 con 0x7fdf6c0b5570 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.446+0000 7fdf720d0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6c0d4f80 con 0x7fdf6c161f20 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.446+0000 7fdf6bfff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf6c0b5c40 0x7fdf6c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.446+0000 7fdf6bfff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf6c0b5c40 0x7fdf6c0d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38492/0 (socket says 172.21.15.7:38492) 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.446+0000 7fdf6bfff640 1 -- 172.21.15.7:0/516382393 learned_addr learned my addr 172.21.15.7:0/516382393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.447+0000 7fdf6affd640 1 --2- 172.21.15.7:0/516382393 >> v2:172.21.15.18:3300/0 conn(0x7fdf6c0b5570 0x7fdf6c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.447+0000 7fdf6b7fe640 1 --2- 172.21.15.7:0/516382393 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 0x7fdf6c0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.447+0000 7fdf6bfff640 1 -- 172.21.15.7:0/516382393 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 msgr2=0x7fdf6c0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.447+0000 7fdf6bfff640 1 --2- 172.21.15.7:0/516382393 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 0x7fdf6c0d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.447+0000 7fdf6bfff640 1 -- 172.21.15.7:0/516382393 >> v2:172.21.15.18:3300/0 conn(0x7fdf6c0b5570 msgr2=0x7fdf6c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.447+0000 7fdf6bfff640 1 --2- 172.21.15.7:0/516382393 >> v2:172.21.15.18:3300/0 conn(0x7fdf6c0b5570 0x7fdf6c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.447+0000 7fdf6bfff640 1 -- 172.21.15.7:0/516382393 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c0d5260 con 0x7fdf6c0b5c40 2024-09-15T23:37:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.447+0000 7fdf6bfff640 1 --2- 172.21.15.7:0/516382393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5c40 0x7fdf6c0d2730 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c08d2f0 tx=0x7fdf6402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75273e2cf25c3ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf6a7fc640 1 -- 172.21.15.7:0/516382393 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64036070 con 0x7fdf6c0b5c40 2024-09-15T23:37:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf6a7fc640 1 -- 172.21.15.7:0/516382393 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf64031600 con 0x7fdf6c0b5c40 2024-09-15T23:37:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf6a7fc640 1 -- 172.21.15.7:0/516382393 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf640319d0 con 0x7fdf6c0b5c40 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf720d0640 1 -- 172.21.15.7:0/516382393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5c40 msgr2=0x7fdf6c0d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf720d0640 1 --2- 172.21.15.7:0/516382393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5c40 0x7fdf6c0d2730 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c08d2f0 tx=0x7fdf6402a6a0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf720d0640 1 -- 172.21.15.7:0/516382393 shutdown_connections 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf720d0640 1 --2- 172.21.15.7:0/516382393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5c40 0x7fdf6c0d2730 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf720d0640 1 --2- 172.21.15.7:0/516382393 >> v2:172.21.15.18:3300/0 conn(0x7fdf6c0b5570 0x7fdf6c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf720d0640 1 --2- 172.21.15.7:0/516382393 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 0x7fdf6c0d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf720d0640 1 -- 172.21.15.7:0/516382393 >> 172.21.15.7:0/516382393 conn(0x7fdf6c08bfd0 msgr2=0x7fdf6c0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf720d0640 1 -- 172.21.15.7:0/516382393 shutdown_connections 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.448+0000 7fdf720d0640 1 -- 172.21.15.7:0/516382393 wait complete. 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.449+0000 7fdf720d0640 1 Processor -- start 2024-09-15T23:37:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.450+0000 7fdf720d0640 1 -- start start 2024-09-15T23:37:40.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.450+0000 7fdf720d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5570 0x7fdf6c14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.450+0000 7fdf720d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf6c0b5c40 0x7fdf6c06e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.450+0000 7fdf720d0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 0x7fdf6c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.450+0000 7fdf720d0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c0b38f0 con 0x7fdf6c0b5570 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.450+0000 7fdf720d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c0d0270 con 0x7fdf6c0b5c40 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.450+0000 7fdf720d0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6c0d2c70 con 0x7fdf6c161f20 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.450+0000 7fdf6b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5570 0x7fdf6c14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5570 0x7fdf6c14e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38500/0 (socket says 172.21.15.7:38500) 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6b7fe640 1 -- 172.21.15.7:0/1943419753 learned_addr learned my addr 172.21.15.7:0/1943419753 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6affd640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf6c0b5c40 0x7fdf6c06e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6bfff640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 0x7fdf6c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6b7fe640 1 -- 172.21.15.7:0/1943419753 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 msgr2=0x7fdf6c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6b7fe640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 0x7fdf6c06e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6b7fe640 1 -- 172.21.15.7:0/1943419753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf6c0b5c40 msgr2=0x7fdf6c06e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6b7fe640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf6c0b5c40 0x7fdf6c06e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6b7fe640 1 -- 172.21.15.7:0/1943419753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c15de90 con 0x7fdf6c0b5570 2024-09-15T23:37:40.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf6b7fe640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5570 0x7fdf6c14e2c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00d5f0 tx=0x7fdf5c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:40.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf53fff640 1 -- 172.21.15.7:0/1943419753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0125f0 con 0x7fdf6c0b5570 2024-09-15T23:37:40.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf53fff640 1 -- 172.21.15.7:0/1943419753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf5c019070 con 0x7fdf6c0b5570 2024-09-15T23:37:40.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf53fff640 1 -- 172.21.15.7:0/1943419753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0129d0 con 0x7fdf6c0b5570 2024-09-15T23:37:40.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c150c40 con 0x7fdf6c0b5570 2024-09-15T23:37:40.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.451+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c151200 con 0x7fdf6c0b5570 2024-09-15T23:37:40.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.452+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6c0d2730 con 0x7fdf6c0b5570 2024-09-15T23:37:40.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.453+0000 7fdf53fff640 1 -- 172.21.15.7:0/1943419753 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 14) ==== 90790+0+0 (secure 0 0 0) 0x7fdf5c012b70 con 0x7fdf6c0b5570 2024-09-15T23:37:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.453+0000 7fdf53fff640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdf5806c220 0x7fdf5806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.453+0000 7fdf53fff640 1 -- 172.21.15.7:0/1943419753 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fdf5c08b5d0 con 0x7fdf6c0b5570 2024-09-15T23:37:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.453+0000 7fdf6affd640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdf5806c220 0x7fdf5806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.454+0000 7fdf6affd640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdf5806c220 0x7fdf5806e6e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c160760 tx=0x7fdf60008040 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.457+0000 7fdf53fff640 1 -- 172.21.15.7:0/1943419753 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf6c15de90 con 0x7fdf6c0b5570 2024-09-15T23:37:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.634+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) -- 0x7fdf6c0b4df0 con 0x7fdf6c0b5570 2024-09-15T23:37:40.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.637+0000 7fdf53fff640 1 -- 172.21.15.7:0/1943419753 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7fdf5c059770 con 0x7fdf6c0b5570 2024-09-15T23:37:40.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdf5806c220 msgr2=0x7fdf5806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdf5806c220 0x7fdf5806e6e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c160760 tx=0x7fdf60008040 comp rx=0 tx=0).stop 2024-09-15T23:37:40.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5570 msgr2=0x7fdf6c14e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5570 0x7fdf6c14e2c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00d5f0 tx=0x7fdf5c00dab0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 shutdown_connections 2024-09-15T23:37:40.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdf5806c220 0x7fdf5806e6e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdf6c161f20 0x7fdf6c06e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf6c0b5c40 0x7fdf6c06e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 --2- 172.21.15.7:0/1943419753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c0b5570 0x7fdf6c14e2c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 >> 172.21.15.7:0/1943419753 conn(0x7fdf6c08bfd0 msgr2=0x7fdf6c15e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:40.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 shutdown_connections 2024-09-15T23:37:40.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.639+0000 7fdf720d0640 1 -- 172.21.15.7:0/1943419753 wait complete. 2024-09-15T23:37:40.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-09-15T23:37:40.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.755+0000 7f7f6b450640 1 Processor -- start 2024-09-15T23:37:40.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.755+0000 7f7f6b450640 1 -- start start 2024-09-15T23:37:40.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f6b450640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f6406f410 0x7f7f640728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f6b450640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f6406f9b0 con 0x7f7f640cfb20 2024-09-15T23:37:40.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f6b450640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f6406fc90 con 0x7f7f640cff00 2024-09-15T23:37:40.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f6b450640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7f6406ff70 con 0x7f7f6406f410 2024-09-15T23:37:40.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f691ec640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7f640cfb20 0x7f7f6415b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57614/0 (socket says 172.21.15.7:57614) 2024-09-15T23:37:40.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f699ed640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f6406f410 0x7f7f640728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f691ec640 1 -- 172.21.15.7:0/4092193146 learned_addr learned my addr 172.21.15.7:0/4092193146 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f699ed640 1 -- 172.21.15.7:0/4092193146 >> v1:172.21.15.18:6789/0 conn(0x7f7f640cff00 legacy=0x7f7f6406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f699ed640 1 -- 172.21.15.7:0/4092193146 >> v1:172.21.15.7:6789/0 conn(0x7f7f640cfb20 legacy=0x7f7f6415b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f5bfff640 1 -- 172.21.15.7:0/4092193146 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1965034766 0 0) 0x7f7f6406f9b0 con 0x7f7f640cfb20 2024-09-15T23:37:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.756+0000 7f7f699ed640 1 -- 172.21.15.7:0/4092193146 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f64070250 con 0x7f7f6406f410 2024-09-15T23:37:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.757+0000 7f7f699ed640 1 --2- 172.21.15.7:0/4092193146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6406f410 0x7f7f640728d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009d90 tx=0x7f7f5c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb20341e11fe3e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.757+0000 7f7f5bfff640 1 -- 172.21.15.7:0/4092193146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c007740 con 0x7f7f6406f410 2024-09-15T23:37:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.757+0000 7f7f5bfff640 1 -- 172.21.15.7:0/4092193146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5c0078e0 con 0x7f7f6406f410 2024-09-15T23:37:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.757+0000 7f7f5bfff640 1 -- 172.21.15.7:0/4092193146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c007cb0 con 0x7f7f6406f410 2024-09-15T23:37:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.758+0000 7f7f6b450640 1 -- 172.21.15.7:0/4092193146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6406f410 msgr2=0x7f7f640728d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.758+0000 7f7f6b450640 1 --2- 172.21.15.7:0/4092193146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6406f410 0x7f7f640728d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009d90 tx=0x7f7f5c02a6f0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.758+0000 7f7f6b450640 1 -- 172.21.15.7:0/4092193146 shutdown_connections 2024-09-15T23:37:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.758+0000 7f7f6b450640 1 --2- 172.21.15.7:0/4092193146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6406f410 0x7f7f640728d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.758+0000 7f7f6b450640 1 -- 172.21.15.7:0/4092193146 >> 172.21.15.7:0/4092193146 conn(0x7f7f640b5f50 msgr2=0x7f7f6406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.758+0000 7f7f6b450640 1 -- 172.21.15.7:0/4092193146 shutdown_connections 2024-09-15T23:37:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.758+0000 7f7f6b450640 1 -- 172.21.15.7:0/4092193146 wait complete. 2024-09-15T23:37:40.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.759+0000 7f7f6b450640 1 Processor -- start 2024-09-15T23:37:40.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.759+0000 7f7f6b450640 1 -- start start 2024-09-15T23:37:40.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f6b450640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f6406f410 0x7f7f6414b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f6b450640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f640cfb20 0x7f7f6415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f6b450640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f640cff00 0x7f7f6415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f6b450640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f64072f90 con 0x7f7f640cfb20 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f6b450640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7f64072e10 con 0x7f7f640cff00 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f6b450640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7f64073110 con 0x7f7f6406f410 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f689eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f640cfb20 0x7f7f6415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f689eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f640cfb20 0x7f7f6415d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38522/0 (socket says 172.21.15.7:38522) 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f689eb640 1 -- 172.21.15.7:0/1122691722 learned_addr learned my addr 172.21.15.7:0/1122691722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f699ed640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f640cff00 0x7f7f6415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f691ec640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f6406f410 0x7f7f6414b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f689eb640 1 -- 172.21.15.7:0/1122691722 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f6406f410 msgr2=0x7f7f6414b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f689eb640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f6406f410 0x7f7f6414b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f689eb640 1 -- 172.21.15.7:0/1122691722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f640cff00 msgr2=0x7f7f6415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f689eb640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f640cff00 0x7f7f6415dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f689eb640 1 -- 172.21.15.7:0/1122691722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f641f61b0 con 0x7f7f640cfb20 2024-09-15T23:37:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f699ed640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f640cff00 0x7f7f6415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.760+0000 7f7f689eb640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f640cfb20 0x7f7f6415d740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400c450 tx=0x7f7f5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.761+0000 7f7f59ffb640 1 -- 172.21.15.7:0/1122691722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f540052d0 con 0x7f7f640cfb20 2024-09-15T23:37:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.761+0000 7f7f59ffb640 1 -- 172.21.15.7:0/1122691722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f54005470 con 0x7f7f640cfb20 2024-09-15T23:37:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.761+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f64147900 con 0x7f7f640cfb20 2024-09-15T23:37:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.761+0000 7f7f59ffb640 1 -- 172.21.15.7:0/1122691722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54012870 con 0x7f7f640cfb20 2024-09-15T23:37:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.761+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6414b880 con 0x7f7f640cfb20 2024-09-15T23:37:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.762+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f640728d0 con 0x7f7f640cfb20 2024-09-15T23:37:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.762+0000 7f7f59ffb640 1 -- 172.21.15.7:0/1122691722 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 14) ==== 90790+0+0 (secure 0 0 0) 0x7f7f54012a10 con 0x7f7f640cfb20 2024-09-15T23:37:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.763+0000 7f7f59ffb640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7f4806c220 0x7f7f4806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.763+0000 7f7f59ffb640 1 -- 172.21.15.7:0/1122691722 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f7f5408aff0 con 0x7f7f640cfb20 2024-09-15T23:37:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.763+0000 7f7f691ec640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7f4806c220 0x7f7f4806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.763+0000 7f7f691ec640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7f4806c220 0x7f7f4806e6e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f60007fd0 tx=0x7f7f60009040 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:40.767+0000 7f7f59ffb640 1 -- 172.21.15.7:0/1122691722 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7f540597a0 con 0x7f7f640cfb20 2024-09-15T23:37:41.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.006+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) -- 0x7f7f64155020 con 0x7f7f640cfb20 2024-09-15T23:37:41.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.703+0000 7f7f59ffb640 1 -- 172.21.15.7:0/1122691722 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v94) ==== 155+0+0 (secure 0 0 0) 0x7f7f540590a0 con 0x7f7f640cfb20 2024-09-15T23:37:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7f4806c220 msgr2=0x7f7f4806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7f4806c220 0x7f7f4806e6e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f60007fd0 tx=0x7f7f60009040 comp rx=0 tx=0).stop 2024-09-15T23:37:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f640cfb20 msgr2=0x7f7f6415d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f640cfb20 0x7f7f6415d740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400c450 tx=0x7f7f5400c910 comp rx=0 tx=0).stop 2024-09-15T23:37:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 shutdown_connections 2024-09-15T23:37:41.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7f4806c220 0x7f7f4806e6e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:41.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f640cff00 0x7f7f6415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:41.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f640cfb20 0x7f7f6415d740 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:41.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 --2- 172.21.15.7:0/1122691722 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f6406f410 0x7f7f6414b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:41.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.706+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 >> 172.21.15.7:0/1122691722 conn(0x7f7f640b5f50 msgr2=0x7f7f640b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:41.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.707+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 shutdown_connections 2024-09-15T23:37:41.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:41.707+0000 7f7f6b450640 1 -- 172.21.15.7:0/1122691722 wait complete. 2024-09-15T23:37:41.723 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2024-09-15T23:37:41.723 INFO:tasks.cephfs_test_runner: 2024-09-15T23:37:41.723 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-15T23:37:41.724 INFO:tasks.cephfs_test_runner:Ran 1 test in 44.599s 2024-09-15T23:37:41.724 INFO:tasks.cephfs_test_runner: 2024-09-15T23:37:41.724 INFO:tasks.cephfs_test_runner:OK 2024-09-15T23:37:41.724 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-15T23:37:41.733 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-15T23:37:41.779 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-15T23:37:41.779 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:41.779 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:41.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056793,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1363 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249344k,nr_inodes=812336,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-15T23:37:41.798 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-15T23:37:41.798 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:41.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056793,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1363 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249344k,nr_inodes=812336,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-15T23:37:41.856 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:41.856 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:41.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1396 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-15T23:37:41.876 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-15T23:37:41.876 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:37:41.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1396 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-15T23:37:41.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:41.936 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-15T23:37:41.955 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-15T23:37:41.955 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-15T23:37:41.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:41.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-15T23:37:42.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T23:37:42.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-15T23:37:42.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.195 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-15T23:37:42.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-15T23:37:42.280 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-09-15T23:37:42.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:37:42.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.282 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-09-15T23:37:42.302 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-15T23:37:42.302 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-15T23:37:42.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.302 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-15T23:37:42.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.374 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-15T23:37:42.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.443 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-09-15T23:37:42.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.539 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-09-15T23:37:42.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:37:42.557 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-09-15T23:37:42.628 INFO:teuthology.orchestra.run.smithi018.stderr:Cannot find device "ceph-brx" 2024-09-15T23:37:42.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:37:42.629 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T23:37:42.638 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T23:37:42.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:37:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.745+0000 7fb2c81a8640 1 Processor -- start 2024-09-15T23:37:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.745+0000 7fb2c81a8640 1 -- start start 2024-09-15T23:37:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.746+0000 7fb2c81a8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 0x7fb2c00b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.746+0000 7fb2c81a8640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb2c00b31c0 0x7fb2c00d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.746+0000 7fb2c81a8640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c006dfa0 con 0x7fb2c00b3bc0 2024-09-15T23:37:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.746+0000 7fb2c81a8640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c0159240 con 0x7fb2c00b31c0 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.746+0000 7fb2c81a8640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c006e210 con 0x7fb2c0158e60 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.746+0000 7fb2c5743640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb2c00b31c0 0x7fb2c00d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.746+0000 7fb2c6745640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fb2c00b3bc0 0x7fb2c006efd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:43744/0 (socket says 172.21.15.7:43744) 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.746+0000 7fb2c6745640 1 -- 172.21.15.7:0/2580941578 learned_addr learned my addr 172.21.15.7:0/2580941578 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.746+0000 7fb2c5f44640 1 --2- 172.21.15.7:0/2580941578 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 0x7fb2c00b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.747+0000 7fb2c5743640 1 -- 172.21.15.7:0/2580941578 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 msgr2=0x7fb2c00b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.747+0000 7fb2c5743640 1 --2- 172.21.15.7:0/2580941578 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 0x7fb2c00b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.747+0000 7fb2c5743640 1 -- 172.21.15.7:0/2580941578 >> v1:172.21.15.18:6789/0 conn(0x7fb2c00b3bc0 legacy=0x7fb2c006efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.747+0000 7fb2c5743640 1 -- 172.21.15.7:0/2580941578 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c006e4f0 con 0x7fb2c00b31c0 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.747+0000 7fb2c5f44640 1 --2- 172.21.15.7:0/2580941578 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 0x7fb2c00b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.747+0000 7fb2c4f42640 1 -- 172.21.15.7:0/2580941578 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1194900696 0 0) 0x7fb2c006dfa0 con 0x7fb2c00b3bc0 2024-09-15T23:37:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.747+0000 7fb2c5743640 1 --2- 172.21.15.7:0/2580941578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2c00b31c0 0x7fb2c00d05f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00097a0 tx=0x7fb2b002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cff2fc9773ba1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c4f42640 1 -- 172.21.15.7:0/2580941578 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0037070 con 0x7fb2c00b31c0 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c4f42640 1 -- 172.21.15.7:0/2580941578 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b0032440 con 0x7fb2c00b31c0 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c4f42640 1 -- 172.21.15.7:0/2580941578 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0032810 con 0x7fb2c00b31c0 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2580941578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2c00b31c0 msgr2=0x7fb2c00d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c81a8640 1 --2- 172.21.15.7:0/2580941578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2c00b31c0 0x7fb2c00d05f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00097a0 tx=0x7fb2b002a210 comp rx=0 tx=0).stop 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2580941578 shutdown_connections 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c81a8640 1 --2- 172.21.15.7:0/2580941578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2c00b31c0 0x7fb2c00d05f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c81a8640 1 --2- 172.21.15.7:0/2580941578 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 0x7fb2c00b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2580941578 >> 172.21.15.7:0/2580941578 conn(0x7fb2c00b21e0 msgr2=0x7fb2c00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2580941578 shutdown_connections 2024-09-15T23:37:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.748+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2580941578 wait complete. 2024-09-15T23:37:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.749+0000 7fb2c81a8640 1 Processor -- start 2024-09-15T23:37:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.749+0000 7fb2c81a8640 1 -- start start 2024-09-15T23:37:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c81a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c00b31c0 0x7fb2c0153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c81a8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2c00b3bc0 0x7fb2c015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c81a8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 0x7fb2c0149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c81a8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c015ca20 con 0x7fb2c00b31c0 2024-09-15T23:37:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c81a8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c006eab0 con 0x7fb2c00b3bc0 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c81a8640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c015cba0 con 0x7fb2c0158e60 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c5f44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c00b31c0 0x7fb2c0153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c5f44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c00b31c0 0x7fb2c0153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38530/0 (socket says 172.21.15.7:38530) 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c5f44640 1 -- 172.21.15.7:0/2742696645 learned_addr learned my addr 172.21.15.7:0/2742696645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c5743640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2c00b3bc0 0x7fb2c015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.750+0000 7fb2c6745640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 0x7fb2c0149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2c5f44640 1 -- 172.21.15.7:0/2742696645 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 msgr2=0x7fb2c0149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2c5f44640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 0x7fb2c0149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2c5f44640 1 -- 172.21.15.7:0/2742696645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2c00b3bc0 msgr2=0x7fb2c015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2c5f44640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2c00b3bc0 0x7fb2c015b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2c5f44640 1 -- 172.21.15.7:0/2742696645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c009b4a0 con 0x7fb2c00b31c0 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2c5f44640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c00b31c0 0x7fb2c0153960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00e960 tx=0x7fb2bc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2b67fc640 1 -- 172.21.15.7:0/2742696645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc017070 con 0x7fb2c00b31c0 2024-09-15T23:37:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2b67fc640 1 -- 172.21.15.7:0/2742696645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc00ca00 con 0x7fb2c00b31c0 2024-09-15T23:37:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2b67fc640 1 -- 172.21.15.7:0/2742696645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc00cdd0 con 0x7fb2c00b31c0 2024-09-15T23:37:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c00b1700 con 0x7fb2c00b31c0 2024-09-15T23:37:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.751+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c0149e20 con 0x7fb2c00b31c0 2024-09-15T23:37:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.752+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c00b3680 con 0x7fb2c00b31c0 2024-09-15T23:37:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.752+0000 7fb2b67fc640 1 -- 172.21.15.7:0/2742696645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fb2bc01a070 con 0x7fb2c00b31c0 2024-09-15T23:37:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.753+0000 7fb2b67fc640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb2a406c2b0 0x7fb2a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.753+0000 7fb2b67fc640 1 -- 172.21.15.7:0/2742696645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fb2bc08b6e0 con 0x7fb2c00b31c0 2024-09-15T23:37:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.753+0000 7fb2c5743640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb2a406c2b0 0x7fb2a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.753+0000 7fb2c5743640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb2a406c2b0 0x7fb2a406e770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0034000 tx=0x7fb2b002e000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.757+0000 7fb2b67fc640 1 -- 172.21.15.7:0/2742696645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2bc059710 con 0x7fb2c00b31c0 2024-09-15T23:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.930+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2c015c110 con 0x7fb2a406c2b0 2024-09-15T23:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.931+0000 7fb2b67fc640 1 -- 172.21.15.7:0/2742696645 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7fb2c015c110 con 0x7fb2a406c2b0 2024-09-15T23:37:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:42.932 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb2a406c2b0 msgr2=0x7fb2a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb2a406c2b0 0x7fb2a406e770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0034000 tx=0x7fb2b002e000 comp rx=0 tx=0).stop 2024-09-15T23:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c00b31c0 msgr2=0x7fb2c0153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c00b31c0 0x7fb2c0153960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00e960 tx=0x7fb2bc00ee20 comp rx=0 tx=0).stop 2024-09-15T23:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 shutdown_connections 2024-09-15T23:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb2a406c2b0 0x7fb2a406e770 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb2c0158e60 0x7fb2c0149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2c00b3bc0 0x7fb2c015b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 --2- 172.21.15.7:0/2742696645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c00b31c0 0x7fb2c0153960 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 >> 172.21.15.7:0/2742696645 conn(0x7fb2c00b21e0 msgr2=0x7fb2c006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 shutdown_connections 2024-09-15T23:37:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:42.933+0000 7fb2c81a8640 1 -- 172.21.15.7:0/2742696645 wait complete. 2024-09-15T23:37:42.945 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-09-15T23:37:41.329266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256064,"kb_used_data":6372,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718656,"statfs":{"total":773094113280,"available":772831903744,"internally_reserved":0,"allocated":6524928,"data_stored":4324615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-732318,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-417,"num_read_kb":-371,"num_write":-355,"num_write_kb":-2902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.994762"},"pg_stats":[{"pgid":"1.0","version":"40'128","reported_seq":247,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-15T23:37:37.913238+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:37:37.913238+0000","last_peered":"2024-09-15T23:37:37.913238+0000","last_clean":"2024-09-15T23:37:37.913238+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:37.913238+0000","last_undegraded":"2024-09-15T23:37:37.913238+0000","last_fullsized":"2024-09-15T23:37:37.913238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1200128,"data_stored":926866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":483328,"data_stored":212614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":803530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":222675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":487424,"data_stored":212771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":503808,"data_stored":223133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":788906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":934120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:37:42.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:37:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.047+0000 7ff4eb853640 1 Processor -- start 2024-09-15T23:37:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.048+0000 7ff4eb853640 1 -- start start 2024-09-15T23:37:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.048+0000 7ff4eb853640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4e415e840 0x7ff4e40d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.048+0000 7ff4eb853640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff4e40d0f90 0x7ff4e40d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.048+0000 7ff4eb853640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e406dd80 con 0x7ff4e40d52e0 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.048+0000 7ff4eb853640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e40cfe90 con 0x7ff4e415e840 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.048+0000 7ff4eb853640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e40d1900 con 0x7ff4e40d0f90 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.048+0000 7ff4e95ef640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4e415e840 0x7ff4e40d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.049+0000 7ff4e95ef640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4e415e840 0x7ff4e40d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38542/0 (socket says 172.21.15.7:38542) 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.049+0000 7ff4e95ef640 1 -- 172.21.15.7:0/3119703769 learned_addr learned my addr 172.21.15.7:0/3119703769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.049+0000 7ff4e95ef640 1 -- 172.21.15.7:0/3119703769 >> v2:172.21.15.18:3300/0 conn(0x7ff4e40d0f90 msgr2=0x7ff4e40d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.049+0000 7ff4e95ef640 1 --2- 172.21.15.7:0/3119703769 >> v2:172.21.15.18:3300/0 conn(0x7ff4e40d0f90 0x7ff4e40d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.049+0000 7ff4e95ef640 1 -- 172.21.15.7:0/3119703769 >> v1:172.21.15.18:6789/0 conn(0x7ff4e40d52e0 legacy=0x7ff4e406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.049+0000 7ff4e95ef640 1 -- 172.21.15.7:0/3119703769 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e40cf560 con 0x7ff4e415e840 2024-09-15T23:37:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4e95ef640 1 --2- 172.21.15.7:0/3119703769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 0x7ff4e40d0320 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40cb620 tx=0x7ff4e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3584ff8e873884e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4dbfff640 1 -- 172.21.15.7:0/3119703769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0037070 con 0x7ff4e415e840 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4dbfff640 1 -- 172.21.15.7:0/3119703769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e0032440 con 0x7ff4e415e840 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4dbfff640 1 -- 172.21.15.7:0/3119703769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0032810 con 0x7ff4e415e840 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4eb853640 1 -- 172.21.15.7:0/3119703769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 msgr2=0x7ff4e40d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4eb853640 1 --2- 172.21.15.7:0/3119703769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 0x7ff4e40d0320 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40cb620 tx=0x7ff4e002a230 comp rx=0 tx=0).stop 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4eb853640 1 -- 172.21.15.7:0/3119703769 shutdown_connections 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4eb853640 1 --2- 172.21.15.7:0/3119703769 >> v2:172.21.15.18:3300/0 conn(0x7ff4e40d0f90 0x7ff4e40d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4eb853640 1 --2- 172.21.15.7:0/3119703769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 0x7ff4e40d0320 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4eb853640 1 -- 172.21.15.7:0/3119703769 >> 172.21.15.7:0/3119703769 conn(0x7ff4e408bfd0 msgr2=0x7ff4e40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4eb853640 1 -- 172.21.15.7:0/3119703769 shutdown_connections 2024-09-15T23:37:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.050+0000 7ff4eb853640 1 -- 172.21.15.7:0/3119703769 wait complete. 2024-09-15T23:37:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.052+0000 7ff4eb853640 1 Processor -- start 2024-09-15T23:37:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.052+0000 7ff4eb853640 1 -- start start 2024-09-15T23:37:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.052+0000 7ff4eb853640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff4e40d0f90 0x7ff4e4157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.052+0000 7ff4eb853640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4e40d52e0 0x7ff4e40cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.052+0000 7ff4eb853640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 0x7ff4e41e6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.052+0000 7ff4eb853640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e406e030 con 0x7ff4e415e840 2024-09-15T23:37:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.052+0000 7ff4eb853640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40cfb20 con 0x7ff4e40d52e0 2024-09-15T23:37:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.052+0000 7ff4eb853640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e415ec20 con 0x7ff4e40d0f90 2024-09-15T23:37:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e9df0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 0x7ff4e41e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e9df0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 0x7ff4e41e6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38550/0 (socket says 172.21.15.7:38550) 2024-09-15T23:37:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e9df0640 1 -- 172.21.15.7:0/3197447793 learned_addr learned my addr 172.21.15.7:0/3197447793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e8dee640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4e40d52e0 0x7ff4e40cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e95ef640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff4e40d0f90 0x7ff4e4157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e9df0640 1 -- 172.21.15.7:0/3197447793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff4e40d0f90 msgr2=0x7ff4e4157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e9df0640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff4e40d0f90 0x7ff4e4157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e9df0640 1 -- 172.21.15.7:0/3197447793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4e40d52e0 msgr2=0x7ff4e40cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e9df0640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4e40d52e0 0x7ff4e40cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e9df0640 1 -- 172.21.15.7:0/3197447793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e41e51d0 con 0x7ff4e415e840 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4e9df0640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 0x7ff4e41e6230 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00b9e0 tx=0x7ff4dc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4d9ffb640 1 -- 172.21.15.7:0/3197447793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc019070 con 0x7ff4e415e840 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4d9ffb640 1 -- 172.21.15.7:0/3197447793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00cac0 con 0x7ff4e415e840 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e41e5470 con 0x7ff4e415e840 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4d9ffb640 1 -- 172.21.15.7:0/3197447793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc014440 con 0x7ff4e415e840 2024-09-15T23:37:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.053+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41e5d10 con 0x7ff4e415e840 2024-09-15T23:37:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.054+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e41e5860 con 0x7ff4e415e840 2024-09-15T23:37:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.055+0000 7ff4d9ffb640 1 -- 172.21.15.7:0/3197447793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7ff4dc01c070 con 0x7ff4e415e840 2024-09-15T23:37:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.055+0000 7ff4d9ffb640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff4c806c2e0 0x7ff4c806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.055+0000 7ff4d9ffb640 1 -- 172.21.15.7:0/3197447793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7ff4dc08b0a0 con 0x7ff4e415e840 2024-09-15T23:37:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.056+0000 7ff4e95ef640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff4c806c2e0 0x7ff4c806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.056+0000 7ff4e95ef640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff4c806c2e0 0x7ff4c806e7a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0034000 tx=0x7ff4e002e000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.060+0000 7ff4d9ffb640 1 -- 172.21.15.7:0/3197447793 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff4dc059160 con 0x7ff4e415e840 2024-09-15T23:37:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.232+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4e408e400 con 0x7ff4c806c2e0 2024-09-15T23:37:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.232+0000 7ff4d9ffb640 1 -- 172.21.15.7:0/3197447793 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7ff4e408e400 con 0x7ff4c806c2e0 2024-09-15T23:37:43.233 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:43.234 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:37:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.234+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff4c806c2e0 msgr2=0x7ff4c806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.234+0000 7ff4eb853640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff4c806c2e0 0x7ff4c806e7a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0034000 tx=0x7ff4e002e000 comp rx=0 tx=0).stop 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.234+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 msgr2=0x7ff4e41e6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.234+0000 7ff4eb853640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 0x7ff4e41e6230 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00b9e0 tx=0x7ff4dc00bea0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.234+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 shutdown_connections 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.234+0000 7ff4eb853640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff4c806c2e0 0x7ff4c806e7a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.234+0000 7ff4eb853640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e415e840 0x7ff4e41e6230 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.234+0000 7ff4eb853640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4e40d52e0 0x7ff4e40cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.234+0000 7ff4eb853640 1 --2- 172.21.15.7:0/3197447793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff4e40d0f90 0x7ff4e4157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.235+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 >> 172.21.15.7:0/3197447793 conn(0x7ff4e408bfd0 msgr2=0x7ff4e40d3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.235+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 shutdown_connections 2024-09-15T23:37:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.235+0000 7ff4eb853640 1 -- 172.21.15.7:0/3197447793 wait complete. 2024-09-15T23:37:43.246 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-09-15T23:37:41.329266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256064,"kb_used_data":6372,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718656,"statfs":{"total":773094113280,"available":772831903744,"internally_reserved":0,"allocated":6524928,"data_stored":4324615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-732318,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-417,"num_read_kb":-371,"num_write":-355,"num_write_kb":-2902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.994762"},"pg_stats":[{"pgid":"1.0","version":"40'128","reported_seq":247,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-15T23:37:37.913238+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:37:37.913238+0000","last_peered":"2024-09-15T23:37:37.913238+0000","last_clean":"2024-09-15T23:37:37.913238+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:37.913238+0000","last_undegraded":"2024-09-15T23:37:37.913238+0000","last_fullsized":"2024-09-15T23:37:37.913238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1200128,"data_stored":926866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":483328,"data_stored":212614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":803530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":222675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":487424,"data_stored":212771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":503808,"data_stored":223133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":788906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":934120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:37:43.247 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T23:37:43.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:37:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.349+0000 7f87d0e1e640 1 Processor -- start 2024-09-15T23:37:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.349+0000 7f87d0e1e640 1 -- start start 2024-09-15T23:37:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.350+0000 7f87d0e1e640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f87cc15a000 0x7f87cc0cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.350+0000 7f87d0e1e640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87cc0ccca0 con 0x7f87cc15d6b0 2024-09-15T23:37:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.350+0000 7f87d0e1e640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87cc1606f0 con 0x7f87cc0d05f0 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.350+0000 7f87d0e1e640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f87cc15a3e0 con 0x7f87cc15a000 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.350+0000 7f87c9d9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f87cc15d6b0 0x7f87cc15ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57626/0 (socket says 172.21.15.7:57626) 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.350+0000 7f87c9d9b640 1 -- 172.21.15.7:0/1151918446 learned_addr learned my addr 172.21.15.7:0/1151918446 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.350+0000 7f87ca59c640 1 --2- 172.21.15.7:0/1151918446 >> v2:172.21.15.18:3300/0 conn(0x7f87cc15a000 0x7f87cc0cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.350+0000 7f87c959a640 1 -- 172.21.15.7:0/1151918446 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3456086433 0 0) 0x7f87cc0ccca0 con 0x7f87cc15d6b0 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.350+0000 7f87c959a640 1 -- 172.21.15.7:0/1151918446 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87b8003520 con 0x7f87cc15d6b0 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.351+0000 7f87c959a640 1 -- 172.21.15.7:0/1151918446 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1639966000 0 0) 0x7f87cc1606f0 con 0x7f87cc0d05f0 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.351+0000 7f87c959a640 1 -- 172.21.15.7:0/1151918446 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87cc0ccca0 con 0x7f87cc0d05f0 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.351+0000 7f87c959a640 1 -- 172.21.15.7:0/1151918446 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 152215625 0 0) 0x7f87b8003520 con 0x7f87cc15d6b0 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.351+0000 7f87ca59c640 1 -- 172.21.15.7:0/1151918446 >> v1:172.21.15.18:6789/0 conn(0x7f87cc0d05f0 legacy=0x7f87cc0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.351+0000 7f87ca59c640 1 -- 172.21.15.7:0/1151918446 >> v1:172.21.15.7:6789/0 conn(0x7f87cc15d6b0 legacy=0x7f87cc15ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.351+0000 7f87ca59c640 1 -- 172.21.15.7:0/1151918446 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc1609c0 con 0x7f87cc15a000 2024-09-15T23:37:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.351+0000 7f87ca59c640 1 --2- 172.21.15.7:0/1151918446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc15a000 0x7f87cc0cd480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009d90 tx=0x7f87c402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46ee546c7640fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87c959a640 1 -- 172.21.15.7:0/1151918446 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4001670 con 0x7f87cc15a000 2024-09-15T23:37:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87c959a640 1 -- 172.21.15.7:0/1151918446 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c4007540 con 0x7f87cc15a000 2024-09-15T23:37:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87c959a640 1 -- 172.21.15.7:0/1151918446 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4007910 con 0x7f87cc15a000 2024-09-15T23:37:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87d0e1e640 1 -- 172.21.15.7:0/1151918446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc15a000 msgr2=0x7f87cc0cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87d0e1e640 1 --2- 172.21.15.7:0/1151918446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc15a000 0x7f87cc0cd480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009d90 tx=0x7f87c402a5f0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87d0e1e640 1 -- 172.21.15.7:0/1151918446 shutdown_connections 2024-09-15T23:37:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87d0e1e640 1 --2- 172.21.15.7:0/1151918446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc15a000 0x7f87cc0cd480 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87d0e1e640 1 -- 172.21.15.7:0/1151918446 >> 172.21.15.7:0/1151918446 conn(0x7f87cc0b21e0 msgr2=0x7f87cc0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87d0e1e640 1 -- 172.21.15.7:0/1151918446 shutdown_connections 2024-09-15T23:37:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.352+0000 7f87d0e1e640 1 -- 172.21.15.7:0/1151918446 wait complete. 2024-09-15T23:37:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87d0e1e640 1 Processor -- start 2024-09-15T23:37:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87d0e1e640 1 -- start start 2024-09-15T23:37:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87d0e1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc0d05f0 0x7f87cc15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87d0e1e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87cc15a000 0x7f87cc071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87d0e1e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc15d6b0 0x7f87cc15cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87d0e1e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0cd9c0 con 0x7f87cc15d6b0 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87d0e1e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0ccf60 con 0x7f87cc0d05f0 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87d0e1e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f87cc0d0a80 con 0x7f87cc15a000 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87cad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc15d6b0 0x7f87cc15cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.354+0000 7f87cad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc15d6b0 0x7f87cc15cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38564/0 (socket says 172.21.15.7:38564) 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87cad9d640 1 -- 172.21.15.7:0/2144477282 learned_addr learned my addr 172.21.15.7:0/2144477282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87ca59c640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc0d05f0 0x7f87cc15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87c9d9b640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87cc15a000 0x7f87cc071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87cad9d640 1 -- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87cc15a000 msgr2=0x7f87cc071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87cad9d640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87cc15a000 0x7f87cc071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87cad9d640 1 -- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc0d05f0 msgr2=0x7f87cc15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87cad9d640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc0d05f0 0x7f87cc15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87cad9d640 1 -- 172.21.15.7:0/2144477282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc1f7b20 con 0x7f87cc15d6b0 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87ca59c640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc0d05f0 0x7f87cc15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87c9d9b640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87cc15a000 0x7f87cc071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87cad9d640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc15d6b0 0x7f87cc15cce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00c400 tx=0x7f87bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87c2ffd640 1 -- 172.21.15.7:0/2144477282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc017070 con 0x7f87cc15d6b0 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87c2ffd640 1 -- 172.21.15.7:0/2144477282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87bc0052d0 con 0x7f87cc15d6b0 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc1f7e00 con 0x7f87cc15d6b0 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87c2ffd640 1 -- 172.21.15.7:0/2144477282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0037f0 con 0x7f87cc15d6b0 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.355+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc1f83c0 con 0x7f87cc15d6b0 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.356+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87cc0d0df0 con 0x7f87cc15d6b0 2024-09-15T23:37:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.357+0000 7f87c2ffd640 1 -- 172.21.15.7:0/2144477282 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f87bc01a070 con 0x7f87cc15d6b0 2024-09-15T23:37:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.357+0000 7f87c2ffd640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f87ac06c2e0 0x7f87ac06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.357+0000 7f87ca59c640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f87ac06c2e0 0x7f87ac06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.357+0000 7f87c2ffd640 1 -- 172.21.15.7:0/2144477282 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f87bc08a7b0 con 0x7f87cc15d6b0 2024-09-15T23:37:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.358+0000 7f87ca59c640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f87ac06c2e0 0x7f87ac06e7a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f87c4034000 tx=0x7f87c402f000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.362+0000 7f87c2ffd640 1 -- 172.21.15.7:0/2144477282 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f87bc0587f0 con 0x7f87cc15d6b0 2024-09-15T23:37:43.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.534+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87cc1f85c0 con 0x7f87ac06c2e0 2024-09-15T23:37:43.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.535+0000 7f87c2ffd640 1 -- 172.21.15.7:0/2144477282 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7f87cc1f85c0 con 0x7f87ac06c2e0 2024-09-15T23:37:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:43.536 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:37:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f87ac06c2e0 msgr2=0x7f87ac06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f87ac06c2e0 0x7f87ac06e7a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f87c4034000 tx=0x7f87c402f000 comp rx=0 tx=0).stop 2024-09-15T23:37:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc15d6b0 msgr2=0x7f87cc15cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc15d6b0 0x7f87cc15cce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00c400 tx=0x7f87bc00c8c0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 shutdown_connections 2024-09-15T23:37:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f87ac06c2e0 0x7f87ac06e7a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc15d6b0 0x7f87cc15cce0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87cc15a000 0x7f87cc071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 --2- 172.21.15.7:0/2144477282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87cc0d05f0 0x7f87cc15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 >> 172.21.15.7:0/2144477282 conn(0x7f87cc0b21e0 msgr2=0x7f87cc15fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 shutdown_connections 2024-09-15T23:37:43.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.537+0000 7f87d0e1e640 1 -- 172.21.15.7:0/2144477282 wait complete. 2024-09-15T23:37:43.548 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-15T23:37:43.329530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256064,"kb_used_data":6372,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718656,"statfs":{"total":773094113280,"available":772831903744,"internally_reserved":0,"allocated":6524928,"data_stored":4324615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-732318,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-417,"num_read_kb":-371,"num_write":-355,"num_write_kb":-2902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.995026"},"pg_stats":[{"pgid":"1.0","version":"40'128","reported_seq":247,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-15T23:37:37.913238+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:37:37.913238+0000","last_peered":"2024-09-15T23:37:37.913238+0000","last_clean":"2024-09-15T23:37:37.913238+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:37.913238+0000","last_undegraded":"2024-09-15T23:37:37.913238+0000","last_fullsized":"2024-09-15T23:37:37.913238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1200128,"data_stored":926866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":483328,"data_stored":212614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":803530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":222675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":487424,"data_stored":212771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":503808,"data_stored":223133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":788906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":934120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:37:43.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:37:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.651+0000 7f2af6f03640 1 Processor -- start 2024-09-15T23:37:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.651+0000 7f2af6f03640 1 -- start start 2024-09-15T23:37:43.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2af6f03640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2af00ceff0 0x7f2af006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2af6f03640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 0x7f2af015ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2af6f03640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af00ce270 con 0x7f2af015aac0 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2af6f03640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2af006fd90 con 0x7f2af00ceff0 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2af6f03640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f2af015ef60 con 0x7f2af015e5f0 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2aeffff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2af00ceff0 0x7f2af006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2aeffff640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2af00ceff0 0x7f2af006f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38570/0 (socket says 172.21.15.7:38570) 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2aeffff640 1 -- 172.21.15.7:0/2415625417 learned_addr learned my addr 172.21.15.7:0/2415625417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2af54a0640 1 --2- 172.21.15.7:0/2415625417 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 0x7f2af015ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2aeffff640 1 -- 172.21.15.7:0/2415625417 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 msgr2=0x7f2af015ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2aeffff640 1 --2- 172.21.15.7:0/2415625417 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 0x7f2af015ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2aeffff640 1 -- 172.21.15.7:0/2415625417 >> v1:172.21.15.18:6789/0 conn(0x7f2af015aac0 legacy=0x7f2af00b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.652+0000 7f2aeffff640 1 -- 172.21.15.7:0/2415625417 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af00ce510 con 0x7f2af00ceff0 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2aeffff640 1 --2- 172.21.15.7:0/2415625417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 0x7f2af006f7b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009870 tx=0x7f2adc02b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f219ccfb332aac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2aef7fe640 1 -- 172.21.15.7:0/2415625417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc038070 con 0x7f2af00ceff0 2024-09-15T23:37:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2aef7fe640 1 -- 172.21.15.7:0/2415625417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc033640 con 0x7f2af00ceff0 2024-09-15T23:37:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2aef7fe640 1 -- 172.21.15.7:0/2415625417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc033a10 con 0x7f2af00ceff0 2024-09-15T23:37:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2af6f03640 1 -- 172.21.15.7:0/2415625417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 msgr2=0x7f2af006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2af6f03640 1 --2- 172.21.15.7:0/2415625417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 0x7f2af006f7b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009870 tx=0x7f2adc02b380 comp rx=0 tx=0).stop 2024-09-15T23:37:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2af6f03640 1 -- 172.21.15.7:0/2415625417 shutdown_connections 2024-09-15T23:37:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2af6f03640 1 --2- 172.21.15.7:0/2415625417 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 0x7f2af015ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2af6f03640 1 --2- 172.21.15.7:0/2415625417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 0x7f2af006f7b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2af6f03640 1 -- 172.21.15.7:0/2415625417 >> 172.21.15.7:0/2415625417 conn(0x7f2af00b21e0 msgr2=0x7f2af00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.653+0000 7f2af6f03640 1 -- 172.21.15.7:0/2415625417 shutdown_connections 2024-09-15T23:37:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.654+0000 7f2af6f03640 1 -- 172.21.15.7:0/2415625417 wait complete. 2024-09-15T23:37:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.655+0000 7f2af6f03640 1 Processor -- start 2024-09-15T23:37:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.655+0000 7f2af6f03640 1 -- start start 2024-09-15T23:37:43.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af6f03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 0x7f2af006c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af6f03640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2af015aac0 0x7f2af0151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af6f03640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 0x7f2af01f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af6f03640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2af015aea0 con 0x7f2af00ceff0 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af6f03640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2af00cead0 con 0x7f2af015aac0 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af6f03640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f2af00d05f0 con 0x7f2af015e5f0 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af4c9f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 0x7f2af006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af4c9f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 0x7f2af006c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38580/0 (socket says 172.21.15.7:38580) 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af4c9f640 1 -- 172.21.15.7:0/2631868431 learned_addr learned my addr 172.21.15.7:0/2631868431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2aeffff640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2af015aac0 0x7f2af0151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af54a0640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 0x7f2af01f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af4c9f640 1 -- 172.21.15.7:0/2631868431 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 msgr2=0x7f2af01f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af4c9f640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 0x7f2af01f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af4c9f640 1 -- 172.21.15.7:0/2631868431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2af015aac0 msgr2=0x7f2af0151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af4c9f640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2af015aac0 0x7f2af0151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af4c9f640 1 -- 172.21.15.7:0/2631868431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af01f3250 con 0x7f2af00ceff0 2024-09-15T23:37:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2af4c9f640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 0x7f2af006c200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c4d0 tx=0x7f2ae400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2aed7fa640 1 -- 172.21.15.7:0/2631868431 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40052d0 con 0x7f2af00ceff0 2024-09-15T23:37:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.656+0000 7f2aed7fa640 1 -- 172.21.15.7:0/2631868431 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae4005470 con 0x7f2af00ceff0 2024-09-15T23:37:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.657+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af015c110 con 0x7f2af00ceff0 2024-09-15T23:37:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.657+0000 7f2aed7fa640 1 -- 172.21.15.7:0/2631868431 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4012850 con 0x7f2af00ceff0 2024-09-15T23:37:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.657+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af015c690 con 0x7f2af00ceff0 2024-09-15T23:37:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.657+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af00d0a10 con 0x7f2af00ceff0 2024-09-15T23:37:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.658+0000 7f2aed7fa640 1 -- 172.21.15.7:0/2631868431 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f2ae40129f0 con 0x7f2af00ceff0 2024-09-15T23:37:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.659+0000 7f2aed7fa640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2ad406c2b0 0x7f2ad406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.659+0000 7f2aeffff640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2ad406c2b0 0x7f2ad406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.659+0000 7f2aed7fa640 1 -- 172.21.15.7:0/2631868431 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f2ae408afc0 con 0x7f2af00ceff0 2024-09-15T23:37:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.659+0000 7f2aeffff640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2ad406c2b0 0x7f2ad406e770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2adc035000 tx=0x7f2adc02f000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.663+0000 7f2aed7fa640 1 -- 172.21.15.7:0/2631868431 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2af01f3250 con 0x7f2af00ceff0 2024-09-15T23:37:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.837+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2af00ae7d0 con 0x7f2af00ceff0 2024-09-15T23:37:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.838+0000 7f2aed7fa640 1 -- 172.21.15.7:0/2631868431 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+14066 (secure 0 0 0) 0x7f2ae4075710 con 0x7f2af00ceff0 2024-09-15T23:37:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:43.839 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"fsid":"9da25f4c-4673-468c-a40c-f01392dffc9b","created":"2024-09-15T23:35:59.576588+0000","modified":"2024-09-15T23:37:39.687567+0000","last_up_change":"2024-09-15T23:36:10.871174+0000","last_in_change":"2024-09-15T23:36:01.705603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:36:09.860411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98ec7d46-317f-41d0-85d3-f3504f146db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6819","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6821","nonce":1578659663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6826","nonce":1578659663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6831","nonce":1578659663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1578659663},{"type":"v1","addr":"172.21.15.7:6829","nonce":1578659663}]},"public_addr":"172.21.15.7:6821/1578659663","cluster_addr":"172.21.15.7:6826/1578659663","heartbeat_back_addr":"172.21.15.7:6831/1578659663","heartbeat_front_addr":"172.21.15.7:6829/1578659663","state":["exists","up"]},{"osd":1,"uuid":"37f65ad9-e883-45f8-9600-013d978d379d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6817","nonce":885929812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6820","nonce":885929812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6825","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6827","nonce":885929812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":885929812},{"type":"v1","addr":"172.21.15.7:6823","nonce":885929812}]},"public_addr":"172.21.15.7:6817/885929812","cluster_addr":"172.21.15.7:6820/885929812","heartbeat_back_addr":"172.21.15.7:6827/885929812","heartbeat_front_addr":"172.21.15.7:6823/885929812","state":["exists","up"]},{"osd":2,"uuid":"31daf586-306d-4234-bc21-cf9833bf3af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6802","nonce":2860639344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6806","nonce":2860639344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6814","nonce":2860639344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2860639344},{"type":"v1","addr":"172.21.15.7:6810","nonce":2860639344}]},"public_addr":"172.21.15.7:6802/2860639344","cluster_addr":"172.21.15.7:6806/2860639344","heartbeat_back_addr":"172.21.15.7:6814/2860639344","heartbeat_front_addr":"172.21.15.7:6810/2860639344","state":["exists","up"]},{"osd":3,"uuid":"8a976294-6121-4529-8794-f62609f9d76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6801","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6803","nonce":2160174572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6807","nonce":2160174572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6815","nonce":2160174572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":2160174572},{"type":"v1","addr":"172.21.15.7:6811","nonce":2160174572}]},"public_addr":"172.21.15.7:6803/2160174572","cluster_addr":"172.21.15.7:6807/2160174572","heartbeat_back_addr":"172.21.15.7:6815/2160174572","heartbeat_front_addr":"172.21.15.7:6811/2160174572","state":["exists","up"]},{"osd":4,"uuid":"a33b8cd1-8470-4e34-9e9c-8c6dd87bb5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6801","nonce":2251983885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6803","nonce":2251983885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6807","nonce":2251983885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2251983885},{"type":"v1","addr":"172.21.15.18:6805","nonce":2251983885}]},"public_addr":"172.21.15.18:6801/2251983885","cluster_addr":"172.21.15.18:6803/2251983885","heartbeat_back_addr":"172.21.15.18:6807/2251983885","heartbeat_front_addr":"172.21.15.18:6805/2251983885","state":["exists","up"]},{"osd":5,"uuid":"a88ba428-0477-4d56-8889-552e0b7229ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6809","nonce":2875348132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6811","nonce":2875348132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6815","nonce":2875348132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2875348132},{"type":"v1","addr":"172.21.15.18:6813","nonce":2875348132}]},"public_addr":"172.21.15.18:6809/2875348132","cluster_addr":"172.21.15.18:6811/2875348132","heartbeat_back_addr":"172.21.15.18:6815/2875348132","heartbeat_front_addr":"172.21.15.18:6813/2875348132","state":["exists","up"]},{"osd":6,"uuid":"023695f4-8073-47f3-94e5-fcec4e708144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6825","nonce":468690220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6827","nonce":468690220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6831","nonce":468690220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":468690220},{"type":"v1","addr":"172.21.15.18:6829","nonce":468690220}]},"public_addr":"172.21.15.18:6825/468690220","cluster_addr":"172.21.15.18:6827/468690220","heartbeat_back_addr":"172.21.15.18:6831/468690220","heartbeat_front_addr":"172.21.15.18:6829/468690220","state":["exists","up"]},{"osd":7,"uuid":"7f04fe2f-99cf-4cde-a003-d38d453334dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6817","nonce":1769482769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6819","nonce":1769482769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6823","nonce":1769482769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1769482769},{"type":"v1","addr":"172.21.15.18:6821","nonce":1769482769}]},"public_addr":"172.21.15.18:6817/1769482769","cluster_addr":"172.21.15.18:6819/1769482769","heartbeat_back_addr":"172.21.15.18:6823/1769482769","heartbeat_front_addr":"172.21.15.18:6821/1769482769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.302825+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.332161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.118198+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.366572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:07.461412+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.714725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.489494+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:36:08.479670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/581925840":"2024-09-16T23:37:37.316248+0000","172.21.15.7:6833/3993537392":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/897775139":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/670254998":"2024-09-16T23:37:30.658739+0000","172.21.15.7:6835/1729414280":"2024-09-16T23:37:36.647033+0000","172.21.15.7:6832/3993537392":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/571759858":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/858111531":"2024-09-16T23:37:37.316248+0000","172.21.15.7:6837/1729414280":"2024-09-16T23:37:36.647033+0000","172.21.15.7:0/991496469":"2024-09-16T23:37:30.658739+0000","172.21.15.7:0/455855372":"2024-09-16T23:37:37.316248+0000","172.21.15.7:6832/4033779377":"2024-09-16T23:37:37.316248+0000","172.21.15.7:0/4150287862":"2024-09-16T23:37:37.316248+0000","172.21.15.7:0/1701209379":"2024-09-16T23:37:37.316248+0000","172.21.15.7:6833/4033779377":"2024-09-16T23:37:37.316248+0000","172.21.15.7:0/4145853352":"2024-09-16T23:37:37.316248+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:37:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2ad406c2b0 msgr2=0x7f2ad406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2ad406c2b0 0x7f2ad406e770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2adc035000 tx=0x7f2adc02f000 comp rx=0 tx=0).stop 2024-09-15T23:37:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 msgr2=0x7f2af006c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 0x7f2af006c200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c4d0 tx=0x7f2ae400c990 comp rx=0 tx=0).stop 2024-09-15T23:37:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 shutdown_connections 2024-09-15T23:37:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2ad406c2b0 0x7f2ad406e770 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2af015e5f0 0x7f2af01f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2af015aac0 0x7f2af0151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 --2- 172.21.15.7:0/2631868431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af00ceff0 0x7f2af006c200 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 >> 172.21.15.7:0/2631868431 conn(0x7f2af00b21e0 msgr2=0x7f2af0158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 shutdown_connections 2024-09-15T23:37:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:43.840+0000 7f2af6f03640 1 -- 172.21.15.7:0/2631868431 wait complete. 2024-09-15T23:37:44.853 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T23:37:44.853 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T23:37:44.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.973+0000 7f656ae2c640 1 Processor -- start 2024-09-15T23:37:44.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.973+0000 7f656ae2c640 1 -- start start 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.974+0000 7f656ae2c640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640d0420 0x7f656406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.974+0000 7f656ae2c640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f65640ce3c0 0x7f65640ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.974+0000 7f656ae2c640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65640cd320 con 0x7f65640cce40 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.974+0000 7f656ae2c640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f656406cb30 con 0x7f65640ce3c0 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.974+0000 7f656ae2c640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f6564146210 con 0x7f65640d0420 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.974+0000 7f6568bc8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640d0420 0x7f656406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.974+0000 7f65693c9640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f65640ce3c0 0x7f65640ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.975+0000 7f65693c9640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f65640ce3c0 0x7f65640ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:41570/0 (socket says 172.21.15.7:41570) 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.975+0000 7f6568bc8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640d0420 0x7f656406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:46800/0 (socket says 172.21.15.7:46800) 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.975+0000 7f65693c9640 1 -- 172.21.15.7:0/3751793905 learned_addr learned my addr 172.21.15.7:0/3751793905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.975+0000 7f6563fff640 1 --1- 172.21.15.7:0/3751793905 >> v1:172.21.15.18:6789/0 conn(0x7f65640cce40 0x7f65640d0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:43794/0 (socket says 172.21.15.7:43794) 2024-09-15T23:37:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.975+0000 7f6568bc8640 1 -- 172.21.15.7:0/3751793905 >> v2:172.21.15.18:3300/0 conn(0x7f65640ce3c0 msgr2=0x7f65640ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.975+0000 7f6568bc8640 1 --2- 172.21.15.7:0/3751793905 >> v2:172.21.15.18:3300/0 conn(0x7f65640ce3c0 0x7f65640ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.975+0000 7f6568bc8640 1 -- 172.21.15.7:0/3751793905 >> v1:172.21.15.18:6789/0 conn(0x7f65640cce40 legacy=0x7f65640d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.975+0000 7f6568bc8640 1 -- 172.21.15.7:0/3751793905 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65640cd600 con 0x7f65640d0420 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.976+0000 7f6568bc8640 1 --2- 172.21.15.7:0/3751793905 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640d0420 0x7f656406c5f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65540097a0 tx=0x7f655402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb1300cb6cf4c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.976+0000 7f65637fe640 1 -- 172.21.15.7:0/3751793905 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554037070 con 0x7f65640d0420 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.976+0000 7f65637fe640 1 -- 172.21.15.7:0/3751793905 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6554032440 con 0x7f65640d0420 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.976+0000 7f65637fe640 1 -- 172.21.15.7:0/3751793905 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554032810 con 0x7f65640d0420 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.976+0000 7f656ae2c640 1 -- 172.21.15.7:0/3751793905 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640d0420 msgr2=0x7f656406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.976+0000 7f656ae2c640 1 --2- 172.21.15.7:0/3751793905 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640d0420 0x7f656406c5f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65540097a0 tx=0x7f655402a210 comp rx=0 tx=0).stop 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.977+0000 7f656ae2c640 1 -- 172.21.15.7:0/3751793905 shutdown_connections 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.977+0000 7f656ae2c640 1 --2- 172.21.15.7:0/3751793905 >> v2:172.21.15.18:3300/0 conn(0x7f65640ce3c0 0x7f65640ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.977+0000 7f656ae2c640 1 --2- 172.21.15.7:0/3751793905 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640d0420 0x7f656406c5f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.977+0000 7f656ae2c640 1 -- 172.21.15.7:0/3751793905 >> 172.21.15.7:0/3751793905 conn(0x7f65640b2fc0 msgr2=0x7f65640b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.977+0000 7f656ae2c640 1 -- 172.21.15.7:0/3751793905 shutdown_connections 2024-09-15T23:37:44.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.977+0000 7f656ae2c640 1 -- 172.21.15.7:0/3751793905 wait complete. 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.978+0000 7f656ae2c640 1 Processor -- start 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.978+0000 7f656ae2c640 1 -- start start 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f656ae2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65640cce40 0x7f656415c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f656ae2c640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640ce3c0 0x7f65640af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f656ae2c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65640d0420 0x7f65640c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f656ae2c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65640cdbc0 con 0x7f65640cce40 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f656ae2c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f65640cdf30 con 0x7f65640d0420 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f656ae2c640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f65640b3d10 con 0x7f65640ce3c0 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6568bc8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65640cce40 0x7f656415c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6568bc8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65640cce40 0x7f656415c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38594/0 (socket says 172.21.15.7:38594) 2024-09-15T23:37:44.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6568bc8640 1 -- 172.21.15.7:0/659710688 learned_addr learned my addr 172.21.15.7:0/659710688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f65693c9640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65640d0420 0x7f65640c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6563fff640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640ce3c0 0x7f65640af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6568bc8640 1 -- 172.21.15.7:0/659710688 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640ce3c0 msgr2=0x7f65640af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6568bc8640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f65640ce3c0 0x7f65640af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6568bc8640 1 -- 172.21.15.7:0/659710688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65640d0420 msgr2=0x7f65640c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6568bc8640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65640d0420 0x7f65640c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6568bc8640 1 -- 172.21.15.7:0/659710688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65640b0c70 con 0x7f65640cce40 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.979+0000 7f6568bc8640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65640cce40 0x7f656415c020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f65540098c0 tx=0x7f65540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.980+0000 7f65617fa640 1 -- 172.21.15.7:0/659710688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554037040 con 0x7f65640cce40 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.980+0000 7f65617fa640 1 -- 172.21.15.7:0/659710688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6554030500 con 0x7f65640cce40 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.980+0000 7f65617fa640 1 -- 172.21.15.7:0/659710688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540308d0 con 0x7f65640cce40 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.980+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65640b0f50 con 0x7f65640cce40 2024-09-15T23:37:44.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.980+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65640ae470 con 0x7f65640cce40 2024-09-15T23:37:44.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.981+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65641e8c40 con 0x7f65640cce40 2024-09-15T23:37:44.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.981+0000 7f65617fa640 1 -- 172.21.15.7:0/659710688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f6554044020 con 0x7f65640cce40 2024-09-15T23:37:44.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.982+0000 7f65617fa640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f655006c2b0 0x7f655006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:44.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.982+0000 7f65617fa640 1 -- 172.21.15.7:0/659710688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f65540ab5b0 con 0x7f65640cce40 2024-09-15T23:37:44.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.982+0000 7f6563fff640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f655006c2b0 0x7f655006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:44.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.982+0000 7f65617fa640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7f6550072ca0 0x7f65500750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:44.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.982+0000 7f65617fa640 1 -- 172.21.15.7:0/659710688 --> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6554032440 con 0x7f6550072ca0 2024-09-15T23:37:44.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.982+0000 7f65617fa640 1 -- 172.21.15.7:0/659710688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f65540b2050 con 0x7f65640cce40 2024-09-15T23:37:44.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.982+0000 7f65693c9640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7f6550072ca0 0x7f65500750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:44.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.983+0000 7f6563fff640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f655006c2b0 0x7f655006e770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f65641f3640 tx=0x7f6558007e80 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:44.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.983+0000 7f65693c9640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7f6550072ca0 0x7f65500750e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:44.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.983+0000 7f65617fa640 1 -- 172.21.15.7:0/659710688 <== osd.0 v2:172.21.15.7:6819/1578659663 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6554032440 con 0x7f6550072ca0 2024-09-15T23:37:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.996+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 --> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f65640ce830 con 0x7f6550072ca0 2024-09-15T23:37:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.996+0000 7f65617fa640 1 -- 172.21.15.7:0/659710688 <== osd.0 v2:172.21.15.7:6819/1578659663 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f65640ce830 con 0x7f6550072ca0 2024-09-15T23:37:44.997 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:44.997 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T23:37:44.997 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7f6550072ca0 msgr2=0x7f65500750e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6819/1578659663,v1:172.21.15.7:6821/1578659663] conn(0x7f6550072ca0 0x7f65500750e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f655006c2b0 msgr2=0x7f655006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f655006c2b0 0x7f655006e770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f65641f3640 tx=0x7f6558007e80 comp rx=0 tx=0).stop 2024-09-15T23:37:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65640cce40 msgr2=0x7f656415c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 --2- 172.21.15.7:0/659710688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65640cce40 0x7f656415c020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f65540098c0 tx=0x7f65540046c0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f65693c9640 1 -- 172.21.15.7:0/659710688 reap_dead start 2024-09-15T23:37:45.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 shutdown_connections 2024-09-15T23:37:45.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 >> 172.21.15.7:0/659710688 conn(0x7f65640b2fc0 msgr2=0x7f65640b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:45.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 shutdown_connections 2024-09-15T23:37:45.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:44.997+0000 7f656ae2c640 1 -- 172.21.15.7:0/659710688 wait complete. 2024-09-15T23:37:45.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T23:37:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.111+0000 7fda5f25e640 1 Processor -- start 2024-09-15T23:37:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.111+0000 7fda5f25e640 1 -- start start 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.112+0000 7fda5f25e640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fda580b5570 0x7fda580b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.112+0000 7fda5f25e640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda580d4f80 con 0x7fda58161f20 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.112+0000 7fda5f25e640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda580d5260 con 0x7fda580d1510 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.112+0000 7fda5f25e640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fda580d5540 con 0x7fda580b5570 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.112+0000 7fda5cffa640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fda58161f20 0x7fda580d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57630/0 (socket says 172.21.15.7:57630) 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.112+0000 7fda5cffa640 1 -- 172.21.15.7:0/694043565 learned_addr learned my addr 172.21.15.7:0/694043565 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.112+0000 7fda5d7fb640 1 --2- 172.21.15.7:0/694043565 >> v2:172.21.15.18:3300/0 conn(0x7fda580b5570 0x7fda580b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.113+0000 7fda577fe640 1 -- 172.21.15.7:0/694043565 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644202541 0 0) 0x7fda580d4f80 con 0x7fda58161f20 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.113+0000 7fda577fe640 1 -- 172.21.15.7:0/694043565 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda40003520 con 0x7fda58161f20 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.113+0000 7fda577fe640 1 -- 172.21.15.7:0/694043565 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207347421 0 0) 0x7fda580d5260 con 0x7fda580d1510 2024-09-15T23:37:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.113+0000 7fda577fe640 1 -- 172.21.15.7:0/694043565 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda580d4f80 con 0x7fda580d1510 2024-09-15T23:37:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.113+0000 7fda5d7fb640 1 -- 172.21.15.7:0/694043565 >> v1:172.21.15.18:6789/0 conn(0x7fda580d1510 legacy=0x7fda580b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.113+0000 7fda577fe640 1 -- 172.21.15.7:0/694043565 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3117525130 0 0) 0x7fda40003520 con 0x7fda58161f20 2024-09-15T23:37:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.113+0000 7fda5d7fb640 1 -- 172.21.15.7:0/694043565 >> v1:172.21.15.7:6789/0 conn(0x7fda58161f20 legacy=0x7fda580d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.113+0000 7fda5d7fb640 1 -- 172.21.15.7:0/694043565 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda580cff90 con 0x7fda580b5570 2024-09-15T23:37:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.114+0000 7fda5d7fb640 1 --2- 172.21.15.7:0/694043565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda580b5570 0x7fda580b4390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda50009d90 tx=0x7fda5002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7333bce8c6a194c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.114+0000 7fda577fe640 1 -- 172.21.15.7:0/694043565 <== mon.1 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48001690 con 0x7fda580b5570 2024-09-15T23:37:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.114+0000 7fda577fe640 1 -- 172.21.15.7:0/694043565 <== mon.1 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda50007540 con 0x7fda580b5570 2024-09-15T23:37:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.114+0000 7fda577fe640 1 -- 172.21.15.7:0/694043565 <== mon.1 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda500078f0 con 0x7fda580b5570 2024-09-15T23:37:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.114+0000 7fda5f25e640 1 -- 172.21.15.7:0/694043565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda580b5570 msgr2=0x7fda580b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.114+0000 7fda5f25e640 1 --2- 172.21.15.7:0/694043565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda580b5570 0x7fda580b4390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda50009d90 tx=0x7fda5002a680 comp rx=0 tx=0).stop 2024-09-15T23:37:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.114+0000 7fda5f25e640 1 -- 172.21.15.7:0/694043565 shutdown_connections 2024-09-15T23:37:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.114+0000 7fda5f25e640 1 --2- 172.21.15.7:0/694043565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda580b5570 0x7fda580b4390 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.114+0000 7fda5f25e640 1 -- 172.21.15.7:0/694043565 >> 172.21.15.7:0/694043565 conn(0x7fda5808bfd0 msgr2=0x7fda580d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.115+0000 7fda5f25e640 1 -- 172.21.15.7:0/694043565 shutdown_connections 2024-09-15T23:37:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.115+0000 7fda5f25e640 1 -- 172.21.15.7:0/694043565 wait complete. 2024-09-15T23:37:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.116+0000 7fda5f25e640 1 Processor -- start 2024-09-15T23:37:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.116+0000 7fda5f25e640 1 -- start start 2024-09-15T23:37:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5f25e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda580b5570 0x7fda58158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5f25e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda580d1510 0x7fda5806e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5f25e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58161f20 0x7fda5806e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5f25e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda580b38f0 con 0x7fda58161f20 2024-09-15T23:37:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5f25e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fda580d4d30 con 0x7fda580d1510 2024-09-15T23:37:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5f25e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fda580b5c40 con 0x7fda580b5570 2024-09-15T23:37:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5d7fb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58161f20 0x7fda5806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5d7fb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58161f20 0x7fda5806e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38604/0 (socket says 172.21.15.7:38604) 2024-09-15T23:37:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5d7fb640 1 -- 172.21.15.7:0/606446204 learned_addr learned my addr 172.21.15.7:0/606446204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda57fff640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda580d1510 0x7fda5806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5cffa640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda580b5570 0x7fda58158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5d7fb640 1 -- 172.21.15.7:0/606446204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda580b5570 msgr2=0x7fda58158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5d7fb640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda580b5570 0x7fda58158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5d7fb640 1 -- 172.21.15.7:0/606446204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda580d1510 msgr2=0x7fda5806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5d7fb640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda580d1510 0x7fda5806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5d7fb640 1 -- 172.21.15.7:0/606446204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda581591d0 con 0x7fda58161f20 2024-09-15T23:37:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.117+0000 7fda5d7fb640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58161f20 0x7fda5806e780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda50009eb0 tx=0x7fda50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.118+0000 7fda557fa640 1 -- 172.21.15.7:0/606446204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50009450 con 0x7fda58161f20 2024-09-15T23:37:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.118+0000 7fda557fa640 1 -- 172.21.15.7:0/606446204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda50004240 con 0x7fda58161f20 2024-09-15T23:37:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.118+0000 7fda557fa640 1 -- 172.21.15.7:0/606446204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50004610 con 0x7fda58161f20 2024-09-15T23:37:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.118+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda580b89a0 con 0x7fda58161f20 2024-09-15T23:37:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.118+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda580b9240 con 0x7fda58161f20 2024-09-15T23:37:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.119+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda580d0e00 con 0x7fda58161f20 2024-09-15T23:37:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.119+0000 7fda557fa640 1 -- 172.21.15.7:0/606446204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fda50039040 con 0x7fda58161f20 2024-09-15T23:37:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.120+0000 7fda557fa640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fda4406c2b0 0x7fda4406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.120+0000 7fda557fa640 1 -- 172.21.15.7:0/606446204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fda500acae0 con 0x7fda58161f20 2024-09-15T23:37:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.120+0000 7fda5cffa640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fda4406c2b0 0x7fda4406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.120+0000 7fda5cffa640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fda4406c2b0 0x7fda4406e770 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda48001800 tx=0x7fda480016b0 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.124+0000 7fda557fa640 1 -- 172.21.15.7:0/606446204 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fda50040030 con 0x7fda58161f20 2024-09-15T23:37:45.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.298+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fda5815f160 con 0x7fda4406c2b0 2024-09-15T23:37:45.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.298+0000 7fda557fa640 1 -- 172.21.15.7:0/606446204 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fda5815f160 con 0x7fda4406c2b0 2024-09-15T23:37:45.299 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T23:37:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fda4406c2b0 msgr2=0x7fda4406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fda4406c2b0 0x7fda4406e770 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda48001800 tx=0x7fda480016b0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58161f20 msgr2=0x7fda5806e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58161f20 0x7fda5806e780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda50009eb0 tx=0x7fda50004a40 comp rx=0 tx=0).stop 2024-09-15T23:37:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 shutdown_connections 2024-09-15T23:37:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fda4406c2b0 0x7fda4406e770 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58161f20 0x7fda5806e780 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda580d1510 0x7fda5806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 --2- 172.21.15.7:0/606446204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fda580b5570 0x7fda58158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 >> 172.21.15.7:0/606446204 conn(0x7fda5808bfd0 msgr2=0x7fda580b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 shutdown_connections 2024-09-15T23:37:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.300+0000 7fda5f25e640 1 -- 172.21.15.7:0/606446204 wait complete. 2024-09-15T23:37:45.312 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T23:37:45.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T23:37:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.418+0000 7f30d5159640 1 Processor -- start 2024-09-15T23:37:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.418+0000 7f30d5159640 1 -- start start 2024-09-15T23:37:45.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30d5159640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30d015e5b0 0x7f30d0162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30d5159640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f30d015e990 0x7f30d00cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30d5159640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d00d2980 con 0x7f30d00d24e0 2024-09-15T23:37:45.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30d5159640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f30d00d2c40 con 0x7f30d015e5b0 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30d5159640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7f30d00d2f20 con 0x7f30d015e990 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30ced9d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30d015e5b0 0x7f30d0162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30ced9d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30d015e5b0 0x7f30d0162350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38612/0 (socket says 172.21.15.7:38612) 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30ced9d640 1 -- 172.21.15.7:0/895116109 learned_addr learned my addr 172.21.15.7:0/895116109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30ced9d640 1 -- 172.21.15.7:0/895116109 >> v2:172.21.15.18:3300/0 conn(0x7f30d015e990 msgr2=0x7f30d00cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30cf59e640 1 --2- 172.21.15.7:0/895116109 >> v2:172.21.15.18:3300/0 conn(0x7f30d015e990 0x7f30d00cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30ced9d640 1 --2- 172.21.15.7:0/895116109 >> v2:172.21.15.18:3300/0 conn(0x7f30d015e990 0x7f30d00cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30ced9d640 1 -- 172.21.15.7:0/895116109 >> v1:172.21.15.18:6789/0 conn(0x7f30d00d24e0 legacy=0x7f30d0162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30ced9d640 1 -- 172.21.15.7:0/895116109 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d00d3200 con 0x7f30d015e5b0 2024-09-15T23:37:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.419+0000 7f30cf59e640 1 --2- 172.21.15.7:0/895116109 >> v2:172.21.15.18:3300/0 conn(0x7f30d015e990 0x7f30d00cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:45.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.420+0000 7f30ced9d640 1 --2- 172.21.15.7:0/895116109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e5b0 0x7f30d0162350 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009870 tx=0x7f30c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5019bf6a54c137 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.420+0000 7f30cdd9b640 1 -- 172.21.15.7:0/895116109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0037070 con 0x7f30d015e5b0 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.420+0000 7f30cdd9b640 1 -- 172.21.15.7:0/895116109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c0032440 con 0x7f30d015e5b0 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.420+0000 7f30cdd9b640 1 -- 172.21.15.7:0/895116109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0032810 con 0x7f30d015e5b0 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.421+0000 7f30d5159640 1 -- 172.21.15.7:0/895116109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e5b0 msgr2=0x7f30d0162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.421+0000 7f30d5159640 1 --2- 172.21.15.7:0/895116109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e5b0 0x7f30d0162350 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009870 tx=0x7f30c002a270 comp rx=0 tx=0).stop 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.421+0000 7f30d5159640 1 -- 172.21.15.7:0/895116109 shutdown_connections 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.421+0000 7f30d5159640 1 --2- 172.21.15.7:0/895116109 >> v2:172.21.15.18:3300/0 conn(0x7f30d015e990 0x7f30d00cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.421+0000 7f30d5159640 1 --2- 172.21.15.7:0/895116109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e5b0 0x7f30d0162350 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.421+0000 7f30d5159640 1 -- 172.21.15.7:0/895116109 >> 172.21.15.7:0/895116109 conn(0x7f30d008bfd0 msgr2=0x7f30d00d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.421+0000 7f30d5159640 1 -- 172.21.15.7:0/895116109 shutdown_connections 2024-09-15T23:37:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.421+0000 7f30d5159640 1 -- 172.21.15.7:0/895116109 wait complete. 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.422+0000 7f30d5159640 1 Processor -- start 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.422+0000 7f30d5159640 1 -- start start 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30d5159640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30d00d24e0 0x7f30d0160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30d5159640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e5b0 0x7f30d015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30d5159640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e990 0x7f30d00b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30d5159640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30d00b3b40 con 0x7f30d015e990 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30d5159640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f30d00b39c0 con 0x7f30d00d24e0 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30d5159640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f30d0162fa0 con 0x7f30d015e5b0 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30cf59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e990 0x7f30d00b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30cf59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e990 0x7f30d00b82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38620/0 (socket says 172.21.15.7:38620) 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30cf59e640 1 -- 172.21.15.7:0/354083283 learned_addr learned my addr 172.21.15.7:0/354083283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30ced9d640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30d00d24e0 0x7f30d0160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.423+0000 7f30ce59c640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e5b0 0x7f30d015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30cf59e640 1 -- 172.21.15.7:0/354083283 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e5b0 msgr2=0x7f30d015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30cf59e640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f30d015e5b0 0x7f30d015ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30cf59e640 1 -- 172.21.15.7:0/354083283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30d00d24e0 msgr2=0x7f30d0160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30cf59e640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30d00d24e0 0x7f30d0160ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30cf59e640 1 -- 172.21.15.7:0/354083283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d00b8c50 con 0x7f30d015e990 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30cf59e640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e990 0x7f30d00b82e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f30c800cd50 tx=0x7f30c800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30b77fe640 1 -- 172.21.15.7:0/354083283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c800f680 con 0x7f30d015e990 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30b77fe640 1 -- 172.21.15.7:0/354083283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c800f840 con 0x7f30d015e990 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d015bd40 con 0x7f30d015e990 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30b77fe640 1 -- 172.21.15.7:0/354083283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c800fc30 con 0x7f30d015e990 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.424+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d015c300 con 0x7f30d015e990 2024-09-15T23:37:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.425+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30d00cb370 con 0x7f30d015e990 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.426+0000 7f30b77fe640 1 -- 172.21.15.7:0/354083283 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f30c800e3e0 con 0x7f30d015e990 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.426+0000 7f30b77fe640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f30bc06c2b0 0x7f30bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.426+0000 7f30b77fe640 1 -- 172.21.15.7:0/354083283 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f30c808c2c0 con 0x7f30d015e990 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.426+0000 7f30ced9d640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f30bc06c2b0 0x7f30bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.426+0000 7f30b77fe640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f30bc072ca0 0x7f30bc0750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.426+0000 7f30b77fe640 1 -- 172.21.15.7:0/354083283 --> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30c8014e50 con 0x7f30bc072ca0 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.426+0000 7f30b77fe640 1 -- 172.21.15.7:0/354083283 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f30c8093050 con 0x7f30d015e990 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.427+0000 7f30ce59c640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f30bc072ca0 0x7f30bc0750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.427+0000 7f30ced9d640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f30bc06c2b0 0x7f30bc06e770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f30c0034000 tx=0x7f30c002e000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.427+0000 7f30ce59c640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f30bc072ca0 0x7f30bc0750e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.427+0000 7f30b77fe640 1 -- 172.21.15.7:0/354083283 <== osd.1 v2:172.21.15.7:6816/885929812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30c8014e50 con 0x7f30bc072ca0 2024-09-15T23:37:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.440+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 --> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f30d0162350 con 0x7f30bc072ca0 2024-09-15T23:37:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.440+0000 7f30b77fe640 1 -- 172.21.15.7:0/354083283 <== osd.1 v2:172.21.15.7:6816/885929812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f30d0162350 con 0x7f30bc072ca0 2024-09-15T23:37:45.441 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T23:37:45.442 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:45.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f30bc072ca0 msgr2=0x7f30bc0750e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6816/885929812,v1:172.21.15.7:6817/885929812] conn(0x7f30bc072ca0 0x7f30bc0750e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f30bc06c2b0 msgr2=0x7f30bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f30bc06c2b0 0x7f30bc06e770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f30c0034000 tx=0x7f30c002e000 comp rx=0 tx=0).stop 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e990 msgr2=0x7f30d00b82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 --2- 172.21.15.7:0/354083283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015e990 0x7f30d00b82e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f30c800cd50 tx=0x7f30c800a650 comp rx=0 tx=0).stop 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30cf59e640 1 -- 172.21.15.7:0/354083283 reap_dead start 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 shutdown_connections 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 >> 172.21.15.7:0/354083283 conn(0x7f30d008bfd0 msgr2=0x7f30d00d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 shutdown_connections 2024-09-15T23:37:45.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.441+0000 7f30d5159640 1 -- 172.21.15.7:0/354083283 wait complete. 2024-09-15T23:37:45.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T23:37:45.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.554+0000 7fb1b2598640 1 Processor -- start 2024-09-15T23:37:45.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.554+0000 7fb1b2598640 1 -- start start 2024-09-15T23:37:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.555+0000 7fb1b2598640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1ac0d1510 0x7fb1ac0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.555+0000 7fb1b2598640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1ac0d4f80 con 0x7fb1ac161f20 2024-09-15T23:37:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.555+0000 7fb1b2598640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1ac0d5260 con 0x7fb1ac0b5570 2024-09-15T23:37:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.555+0000 7fb1b2598640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb1ac0d5540 con 0x7fb1ac0d1510 2024-09-15T23:37:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.555+0000 7fb1abfff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb1ac161f20 0x7fb1ac0d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57634/0 (socket says 172.21.15.7:57634) 2024-09-15T23:37:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.555+0000 7fb1abfff640 1 -- 172.21.15.7:0/2406546909 learned_addr learned my addr 172.21.15.7:0/2406546909 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1ab7fe640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1ac0d1510 0x7fb1ac0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056984702 0 0) 0x7fb1ac0d4f80 con 0x7fb1ac161f20 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb194003520 con 0x7fb1ac161f20 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1249511444 0 0) 0x7fb1ac0d5260 con 0x7fb1ac0b5570 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1ac0d4f80 con 0x7fb1ac0b5570 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3475175396 0 0) 0x7fb194003520 con 0x7fb1ac161f20 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1ac0d1510 msgr2=0x7fb1ac0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1ac0d1510 0x7fb1ac0b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 >> v1:172.21.15.18:6789/0 conn(0x7fb1ac0b5570 legacy=0x7fb1ac0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac0cff90 con 0x7fb1ac161f20 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7fb19c001670 con 0x7fb1ac161f20 2024-09-15T23:37:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 >> v1:172.21.15.7:6789/0 conn(0x7fb1ac161f20 legacy=0x7fb1ac0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb194003b70 0x7fb194003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb194004800 0x7fb194004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1940051c0 0x7fb19400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb19400bfb0 con 0x7fb1940051c0 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb19400c160 con 0x7fb194004800 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb19400c360 con 0x7fb194003b70 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1b0b35640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1940051c0 0x7fb19400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1abfff640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb194003b70 0x7fb194003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1ab7fe640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb194004800 0x7fb194004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1b0b35640 1 -- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb194003b70 msgr2=0x7fb194003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1b0b35640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb194003b70 0x7fb194003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1b0b35640 1 -- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb194004800 msgr2=0x7fb194004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1b0b35640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb194004800 0x7fb194004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.556+0000 7fb1b0b35640 1 -- 172.21.15.7:0/2406546909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb194003520 con 0x7fb1940051c0 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.557+0000 7fb1b0b35640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1940051c0 0x7fb19400b9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4001fd0 tx=0x7fb194005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b848e1f510b47a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.557+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4001bf0 con 0x7fb1940051c0 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.557+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a4003b70 con 0x7fb1940051c0 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.557+0000 7fb1aaffd640 1 -- 172.21.15.7:0/2406546909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40050d0 con 0x7fb1940051c0 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.558+0000 7fb1b2598640 1 -- 172.21.15.7:0/2406546909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1940051c0 msgr2=0x7fb19400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.558+0000 7fb1b2598640 1 --2- 172.21.15.7:0/2406546909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1940051c0 0x7fb19400b9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4001fd0 tx=0x7fb194005830 comp rx=0 tx=0).stop 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.558+0000 7fb1b0b35640 1 -- 172.21.15.7:0/2406546909 reap_dead start 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.558+0000 7fb1b2598640 1 -- 172.21.15.7:0/2406546909 shutdown_connections 2024-09-15T23:37:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.558+0000 7fb1b2598640 1 -- 172.21.15.7:0/2406546909 >> 172.21.15.7:0/2406546909 conn(0x7fb1ac08bfd0 msgr2=0x7fb1ac0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:45.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.558+0000 7fb1b2598640 1 -- 172.21.15.7:0/2406546909 shutdown_connections 2024-09-15T23:37:45.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.558+0000 7fb1b2598640 1 -- 172.21.15.7:0/2406546909 wait complete. 2024-09-15T23:37:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.559+0000 7fb1b2598640 1 Processor -- start 2024-09-15T23:37:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.559+0000 7fb1b2598640 1 -- start start 2024-09-15T23:37:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1b2598640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1940051c0 0x7fb1ac0a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1b2598640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1ac15aab0 0x7fb1ac0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1b2598640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1ac15af30 0x7fb1ac0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1b2598640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac0b38f0 con 0x7fb1ac15aab0 2024-09-15T23:37:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1b2598640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac0d4d30 con 0x7fb1ac15af30 2024-09-15T23:37:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1b2598640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb1ac0b5c40 con 0x7fb1940051c0 2024-09-15T23:37:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1ab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1ac15aab0 0x7fb1ac0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1ab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1ac15aab0 0x7fb1ac0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38630/0 (socket says 172.21.15.7:38630) 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1ab7fe640 1 -- 172.21.15.7:0/2712634899 learned_addr learned my addr 172.21.15.7:0/2712634899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1abfff640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1940051c0 0x7fb1ac0a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1b0b35640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1ac15af30 0x7fb1ac0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1ab7fe640 1 -- 172.21.15.7:0/2712634899 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1940051c0 msgr2=0x7fb1ac0a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1ab7fe640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1940051c0 0x7fb1ac0a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1ab7fe640 1 -- 172.21.15.7:0/2712634899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1ac15af30 msgr2=0x7fb1ac0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1ab7fe640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1ac15af30 0x7fb1ac0cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1ab7fe640 1 -- 172.21.15.7:0/2712634899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac0b81d0 con 0x7fb1ac15aab0 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.560+0000 7fb1b0b35640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1ac15af30 0x7fb1ac0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.561+0000 7fb1ab7fe640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1ac15aab0 0x7fb1ac0cd900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00119d0 tx=0x7fb1a0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.561+0000 7fb1a8ff9640 1 -- 172.21.15.7:0/2712634899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000faf0 con 0x7fb1ac15aab0 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.561+0000 7fb1a8ff9640 1 -- 172.21.15.7:0/2712634899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a000fc90 con 0x7fb1ac15aab0 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.561+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac0b84b0 con 0x7fb1ac15aab0 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.561+0000 7fb1a8ff9640 1 -- 172.21.15.7:0/2712634899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0014440 con 0x7fb1ac15aab0 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.561+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac0b8a70 con 0x7fb1ac15aab0 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.562+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ac0d5260 con 0x7fb1ac15aab0 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.562+0000 7fb1a8ff9640 1 -- 172.21.15.7:0/2712634899 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fb1a0012040 con 0x7fb1ac15aab0 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.563+0000 7fb1a8ff9640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb19806c2b0 0x7fb19806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.563+0000 7fb1a8ff9640 1 -- 172.21.15.7:0/2712634899 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fb1a008bb20 con 0x7fb1ac15aab0 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.563+0000 7fb1abfff640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb19806c2b0 0x7fb19806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.564+0000 7fb1abfff640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb19806c2b0 0x7fb19806e770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb19c001eb0 tx=0x7fb19c009040 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.567+0000 7fb1a8ff9640 1 -- 172.21.15.7:0/2712634899 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb1a0019030 con 0x7fb1ac15aab0 2024-09-15T23:37:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.741+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fb1ac0b5200 con 0x7fb19806c2b0 2024-09-15T23:37:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.742+0000 7fb1a8ff9640 1 -- 172.21.15.7:0/2712634899 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb1ac0b5200 con 0x7fb19806c2b0 2024-09-15T23:37:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T23:37:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb19806c2b0 msgr2=0x7fb19806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb19806c2b0 0x7fb19806e770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb19c001eb0 tx=0x7fb19c009040 comp rx=0 tx=0).stop 2024-09-15T23:37:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1ac15aab0 msgr2=0x7fb1ac0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1ac15aab0 0x7fb1ac0cd900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00119d0 tx=0x7fb1a0011e90 comp rx=0 tx=0).stop 2024-09-15T23:37:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 shutdown_connections 2024-09-15T23:37:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb19806c2b0 0x7fb19806e770 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1ac15af30 0x7fb1ac0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1ac15aab0 0x7fb1ac0cd900 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 --2- 172.21.15.7:0/2712634899 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1940051c0 0x7fb1ac0a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 >> 172.21.15.7:0/2712634899 conn(0x7fb1ac08bfd0 msgr2=0x7fb1ac0b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:45.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 shutdown_connections 2024-09-15T23:37:45.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.744+0000 7fb1b2598640 1 -- 172.21.15.7:0/2712634899 wait complete. 2024-09-15T23:37:45.755 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T23:37:45.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T23:37:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.861+0000 7f2d813fd640 1 Processor -- start 2024-09-15T23:37:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.861+0000 7f2d813fd640 1 -- start start 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.862+0000 7f2d813fd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d7c161e20 0x7f2d7c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.862+0000 7f2d813fd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7c0702d0 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.862+0000 7f2d813fd640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7c0b64a0 con 0x7f2d7c06c320 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.862+0000 7f2d813fd640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2d7c162200 con 0x7f2d7c161e20 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.862+0000 7f2d7affd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d7c161e20 0x7f2d7c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.862+0000 7f2d7a7fc640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2d7c0b03f0 0x7f2d7c06bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57642/0 (socket says 172.21.15.7:57642) 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.862+0000 7f2d7affd640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d7c161e20 0x7f2d7c09aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38644/0 (socket says 172.21.15.7:38644) 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.862+0000 7f2d7a7fc640 1 -- 172.21.15.7:0/705199804 learned_addr learned my addr 172.21.15.7:0/705199804 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.863+0000 7f2d79ffb640 1 -- 172.21.15.7:0/705199804 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405807220 0 0) 0x7f2d7c0702d0 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.863+0000 7f2d79ffb640 1 -- 172.21.15.7:0/705199804 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d68003520 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.863+0000 7f2d7affd640 1 -- 172.21.15.7:0/705199804 >> v1:172.21.15.18:6789/0 conn(0x7f2d7c06c320 legacy=0x7f2d7c0ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.863+0000 7f2d7affd640 1 -- 172.21.15.7:0/705199804 >> v1:172.21.15.7:6789/0 conn(0x7f2d7c0b03f0 legacy=0x7f2d7c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.863+0000 7f2d7affd640 1 -- 172.21.15.7:0/705199804 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c070570 con 0x7f2d7c161e20 2024-09-15T23:37:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d7affd640 1 --2- 172.21.15.7:0/705199804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c161e20 0x7f2d7c09aa40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009d90 tx=0x7f2d6c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1e0797f29250aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d79ffb640 1 -- 172.21.15.7:0/705199804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c007740 con 0x7f2d7c161e20 2024-09-15T23:37:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d79ffb640 1 -- 172.21.15.7:0/705199804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c0078e0 con 0x7f2d7c161e20 2024-09-15T23:37:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d79ffb640 1 -- 172.21.15.7:0/705199804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c007cb0 con 0x7f2d7c161e20 2024-09-15T23:37:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d813fd640 1 -- 172.21.15.7:0/705199804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c161e20 msgr2=0x7f2d7c09aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d813fd640 1 --2- 172.21.15.7:0/705199804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c161e20 0x7f2d7c09aa40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009d90 tx=0x7f2d6c02a6f0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d813fd640 1 -- 172.21.15.7:0/705199804 shutdown_connections 2024-09-15T23:37:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d813fd640 1 --2- 172.21.15.7:0/705199804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c161e20 0x7f2d7c09aa40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d813fd640 1 -- 172.21.15.7:0/705199804 >> 172.21.15.7:0/705199804 conn(0x7f2d7c0734b0 msgr2=0x7f2d7c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d813fd640 1 -- 172.21.15.7:0/705199804 shutdown_connections 2024-09-15T23:37:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.864+0000 7f2d813fd640 1 -- 172.21.15.7:0/705199804 wait complete. 2024-09-15T23:37:45.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.866+0000 7f2d813fd640 1 Processor -- start 2024-09-15T23:37:45.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.866+0000 7f2d813fd640 1 -- start start 2024-09-15T23:37:45.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.866+0000 7f2d813fd640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d7c06c320 0x7f2d7c1608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.866+0000 7f2d813fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c0b03f0 0x7f2d7c15fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.866+0000 7f2d813fd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d7c161e20 0x7f2d7c1f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.866+0000 7f2d813fd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0b60c0 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.866+0000 7f2d813fd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0b6780 con 0x7f2d7c161e20 2024-09-15T23:37:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.866+0000 7f2d813fd640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7c0b6240 con 0x7f2d7c06c320 2024-09-15T23:37:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c0b03f0 0x7f2d7c15fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c0b03f0 0x7f2d7c15fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38656/0 (socket says 172.21.15.7:38656) 2024-09-15T23:37:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7a7fc640 1 -- 172.21.15.7:0/1721499639 learned_addr learned my addr 172.21.15.7:0/1721499639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7affd640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d7c06c320 0x7f2d7c1608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7b7fe640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d7c161e20 0x7f2d7c1f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7a7fc640 1 -- 172.21.15.7:0/1721499639 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d7c06c320 msgr2=0x7f2d7c1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7a7fc640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d7c06c320 0x7f2d7c1608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7a7fc640 1 -- 172.21.15.7:0/1721499639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d7c161e20 msgr2=0x7f2d7c1f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7a7fc640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d7c161e20 0x7f2d7c1f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7a7fc640 1 -- 172.21.15.7:0/1721499639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c1f5020 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7b7fe640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d7c161e20 0x7f2d7c1f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7affd640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d7c06c320 0x7f2d7c1608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d7a7fc640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c0b03f0 0x7f2d7c15fbb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2d64001eb0 tx=0x7f2d6400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d737fe640 1 -- 172.21.15.7:0/1721499639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64017070 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d737fe640 1 -- 172.21.15.7:0/1721499639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d64005ce0 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c1f4780 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d737fe640 1 -- 172.21.15.7:0/1721499639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64007270 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.867+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c1f4d40 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.868+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d7c160e20 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.869+0000 7f2d737fe640 1 -- 172.21.15.7:0/1721499639 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f2d6401a070 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.869+0000 7f2d737fe640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2d5c06c2b0 0x7f2d5c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.869+0000 7f2d737fe640 1 -- 172.21.15.7:0/1721499639 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f2d6408b2c0 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.870+0000 7f2d7affd640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2d5c06c2b0 0x7f2d5c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.870+0000 7f2d737fe640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f2d5c072ca0 0x7f2d5c0750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.870+0000 7f2d737fe640 1 -- 172.21.15.7:0/1721499639 --> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d64024ea0 con 0x7f2d5c072ca0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.870+0000 7f2d737fe640 1 -- 172.21.15.7:0/1721499639 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2d6408b750 con 0x7f2d7c0b03f0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.870+0000 7f2d7b7fe640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f2d5c072ca0 0x7f2d5c0750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.870+0000 7f2d7affd640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2d5c06c2b0 0x7f2d5c06e770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c034000 tx=0x7f2d6c033040 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.870+0000 7f2d7b7fe640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f2d5c072ca0 0x7f2d5c0750e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:45.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.871+0000 7f2d737fe640 1 -- 172.21.15.7:0/1721499639 <== osd.2 v2:172.21.15.7:6800/2860639344 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d64024ea0 con 0x7f2d5c072ca0 2024-09-15T23:37:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.883+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 --> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2d7c1f54c0 con 0x7f2d5c072ca0 2024-09-15T23:37:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d737fe640 1 -- 172.21.15.7:0/1721499639 <== osd.2 v2:172.21.15.7:6800/2860639344 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2d7c1f54c0 con 0x7f2d5c072ca0 2024-09-15T23:37:45.884 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T23:37:45.885 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:45.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f2d5c072ca0 msgr2=0x7f2d5c0750e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6800/2860639344,v1:172.21.15.7:6802/2860639344] conn(0x7f2d5c072ca0 0x7f2d5c0750e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2d5c06c2b0 msgr2=0x7f2d5c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f2d5c06c2b0 0x7f2d5c06e770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c034000 tx=0x7f2d6c033040 comp rx=0 tx=0).stop 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c0b03f0 msgr2=0x7f2d7c15fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 --2- 172.21.15.7:0/1721499639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7c0b03f0 0x7f2d7c15fbb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2d64001eb0 tx=0x7f2d6400c960 comp rx=0 tx=0).stop 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d7b7fe640 1 -- 172.21.15.7:0/1721499639 reap_dead start 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 shutdown_connections 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 >> 172.21.15.7:0/1721499639 conn(0x7f2d7c0734b0 msgr2=0x7f2d7c0b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 shutdown_connections 2024-09-15T23:37:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.884+0000 7f2d813fd640 1 -- 172.21.15.7:0/1721499639 wait complete. 2024-09-15T23:37:45.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T23:37:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.998+0000 7fe99619f640 1 Processor -- start 2024-09-15T23:37:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.998+0000 7fe99619f640 1 -- start start 2024-09-15T23:37:46.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.999+0000 7fe99619f640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe99006f790 0x7fe990072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.999+0000 7fe99619f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900c5720 0x7fe9900c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.999+0000 7fe99619f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9900cfe20 con 0x7fe99006fb70 2024-09-15T23:37:46.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.999+0000 7fe99619f640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe9900721b0 con 0x7fe99006f790 2024-09-15T23:37:46.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.999+0000 7fe99619f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe990072490 con 0x7fe9900c5720 2024-09-15T23:37:46.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.999+0000 7fe99519d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe99006f790 0x7fe990072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.999+0000 7fe99499c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe99006fb70 0x7fe9900cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57646/0 (socket says 172.21.15.7:57646) 2024-09-15T23:37:46.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.999+0000 7fe99499c640 1 -- 172.21.15.7:0/344639711 learned_addr learned my addr 172.21.15.7:0/344639711 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:46.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:45.999+0000 7fe99599e640 1 --2- 172.21.15.7:0/344639711 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900c5720 0x7fe9900c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.000+0000 7fe99519d640 1 -- 172.21.15.7:0/344639711 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900c5720 msgr2=0x7fe9900c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:46.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.000+0000 7fe99519d640 1 --2- 172.21.15.7:0/344639711 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900c5720 0x7fe9900c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.000+0000 7fe99519d640 1 -- 172.21.15.7:0/344639711 >> v1:172.21.15.7:6789/0 conn(0x7fe99006fb70 legacy=0x7fe9900cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.000+0000 7fe98ffff640 1 -- 172.21.15.7:0/344639711 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 102784918 0 0) 0x7fe9900cfe20 con 0x7fe99006fb70 2024-09-15T23:37:46.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.000+0000 7fe99519d640 1 -- 172.21.15.7:0/344639711 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990072770 con 0x7fe99006f790 2024-09-15T23:37:46.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.000+0000 7fe99519d640 1 --2- 172.21.15.7:0/344639711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006f790 0x7fe990072f70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe980009d90 tx=0x7fe98002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2289a22c6d244 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.000+0000 7fe98ffff640 1 -- 172.21.15.7:0/344639711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980007740 con 0x7fe99006f790 2024-09-15T23:37:46.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.000+0000 7fe98ffff640 1 -- 172.21.15.7:0/344639711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9800078e0 con 0x7fe99006f790 2024-09-15T23:37:46.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.000+0000 7fe98ffff640 1 -- 172.21.15.7:0/344639711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980007cb0 con 0x7fe99006f790 2024-09-15T23:37:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.001+0000 7fe99619f640 1 -- 172.21.15.7:0/344639711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006f790 msgr2=0x7fe990072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.001+0000 7fe99619f640 1 --2- 172.21.15.7:0/344639711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006f790 0x7fe990072f70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe980009d90 tx=0x7fe98002a660 comp rx=0 tx=0).stop 2024-09-15T23:37:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.001+0000 7fe99619f640 1 -- 172.21.15.7:0/344639711 shutdown_connections 2024-09-15T23:37:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.001+0000 7fe99619f640 1 --2- 172.21.15.7:0/344639711 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900c5720 0x7fe9900c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.001+0000 7fe99619f640 1 --2- 172.21.15.7:0/344639711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006f790 0x7fe990072f70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.001+0000 7fe99619f640 1 -- 172.21.15.7:0/344639711 >> 172.21.15.7:0/344639711 conn(0x7fe990070670 msgr2=0x7fe99006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.001+0000 7fe99619f640 1 -- 172.21.15.7:0/344639711 shutdown_connections 2024-09-15T23:37:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.001+0000 7fe99619f640 1 -- 172.21.15.7:0/344639711 wait complete. 2024-09-15T23:37:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.002+0000 7fe99619f640 1 Processor -- start 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.002+0000 7fe99619f640 1 -- start start 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99619f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe99006f790 0x7fe990151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99619f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006fb70 0x7fe99014f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99619f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9900c5720 0x7fe9900cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99619f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe99015d5f0 con 0x7fe99006fb70 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99619f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900cfb20 con 0x7fe9900c5720 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99619f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe99015d770 con 0x7fe99006f790 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99499c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006fb70 0x7fe99014f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99499c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006fb70 0x7fe99014f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38684/0 (socket says 172.21.15.7:38684) 2024-09-15T23:37:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99499c640 1 -- 172.21.15.7:0/596231204 learned_addr learned my addr 172.21.15.7:0/596231204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99599e640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9900c5720 0x7fe9900cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99519d640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe99006f790 0x7fe990151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99499c640 1 -- 172.21.15.7:0/596231204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe99006f790 msgr2=0x7fe990151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99499c640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe99006f790 0x7fe990151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99499c640 1 -- 172.21.15.7:0/596231204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9900c5720 msgr2=0x7fe9900cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99499c640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9900c5720 0x7fe9900cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.003+0000 7fe99499c640 1 -- 172.21.15.7:0/596231204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990150820 con 0x7fe99006fb70 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.004+0000 7fe99499c640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006fb70 0x7fe99014f060 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe98400c470 tx=0x7fe98400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.004+0000 7fe98dffb640 1 -- 172.21.15.7:0/596231204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984018070 con 0x7fe99006fb70 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.004+0000 7fe98dffb640 1 -- 172.21.15.7:0/596231204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9840074e0 con 0x7fe99006fb70 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.004+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe990151da0 con 0x7fe99006fb70 2024-09-15T23:37:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.004+0000 7fe98dffb640 1 -- 172.21.15.7:0/596231204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984004a20 con 0x7fe99006fb70 2024-09-15T23:37:46.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.004+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe990150d80 con 0x7fe99006fb70 2024-09-15T23:37:46.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.005+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe990072f70 con 0x7fe99006fb70 2024-09-15T23:37:46.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.005+0000 7fe98dffb640 1 -- 172.21.15.7:0/596231204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fe98401b070 con 0x7fe99006fb70 2024-09-15T23:37:46.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.006+0000 7fe98dffb640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.006+0000 7fe98dffb640 1 -- 172.21.15.7:0/596231204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fe98408a6a0 con 0x7fe99006fb70 2024-09-15T23:37:46.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.006+0000 7fe99519d640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.006+0000 7fe99519d640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe980034000 tx=0x7fe98002f000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.010+0000 7fe98dffb640 1 -- 172.21.15.7:0/596231204 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe990150820 con 0x7fe99006fb70 2024-09-15T23:37:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.185+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fe9900cf640 con 0x7fe97c06c2e0 2024-09-15T23:37:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.185+0000 7fe98dffb640 1 -- 172.21.15.7:0/596231204 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe9900cf640 con 0x7fe97c06c2e0 2024-09-15T23:37:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T23:37:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe97c06c2e0 msgr2=0x7fe97c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe980034000 tx=0x7fe98002f000 comp rx=0 tx=0).stop 2024-09-15T23:37:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006fb70 msgr2=0x7fe99014f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006fb70 0x7fe99014f060 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe98400c470 tx=0x7fe98400c930 comp rx=0 tx=0).stop 2024-09-15T23:37:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 shutdown_connections 2024-09-15T23:37:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9900c5720 0x7fe9900cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99006fb70 0x7fe99014f060 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 --2- 172.21.15.7:0/596231204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe99006f790 0x7fe990151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.187+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 >> 172.21.15.7:0/596231204 conn(0x7fe990070670 msgr2=0x7fe990069bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.188+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 shutdown_connections 2024-09-15T23:37:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.188+0000 7fe99619f640 1 -- 172.21.15.7:0/596231204 wait complete. 2024-09-15T23:37:46.199 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T23:37:46.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T23:37:46.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.305+0000 7ff269096640 1 Processor -- start 2024-09-15T23:37:46.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.305+0000 7ff269096640 1 -- start start 2024-09-15T23:37:46.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff269096640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff2641558e0 0x7ff2640ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff269096640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff26415f600 0x7ff264072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff269096640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2640af270 con 0x7ff26406e060 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff269096640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff2640b3160 con 0x7ff26415f600 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff269096640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff2640af550 con 0x7ff2641558e0 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff262d9d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff2641558e0 0x7ff2640ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff26359e640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff26415f600 0x7ff264072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff26259c640 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7ff26406e060 0x7ff26415eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.7:43814/0 (socket says 172.21.15.7:43814) 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff262d9d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff2641558e0 0x7ff2640ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.7:46896/0 (socket says 172.21.15.7:46896) 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff26259c640 1 -- 172.21.15.7:0/3400011971 learned_addr learned my addr 172.21.15.7:0/3400011971 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.306+0000 7ff26359e640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff26415f600 0x7ff264072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.7:41670/0 (socket says 172.21.15.7:41670) 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.307+0000 7ff261d9b640 1 -- 172.21.15.7:0/3400011971 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705051647 0 0) 0x7ff2640af270 con 0x7ff26406e060 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.307+0000 7ff262d9d640 1 -- 172.21.15.7:0/3400011971 >> v2:172.21.15.18:3300/0 conn(0x7ff26415f600 msgr2=0x7ff264072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.307+0000 7ff262d9d640 1 --2- 172.21.15.7:0/3400011971 >> v2:172.21.15.18:3300/0 conn(0x7ff26415f600 0x7ff264072b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.307+0000 7ff262d9d640 1 -- 172.21.15.7:0/3400011971 >> v1:172.21.15.18:6789/0 conn(0x7ff26406e060 legacy=0x7ff26415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.307+0000 7ff262d9d640 1 -- 172.21.15.7:0/3400011971 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff264072020 con 0x7ff2641558e0 2024-09-15T23:37:46.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.307+0000 7ff26359e640 1 --2- 172.21.15.7:0/3400011971 >> v2:172.21.15.18:3300/0 conn(0x7ff26415f600 0x7ff264072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.307+0000 7ff262d9d640 1 --2- 172.21.15.7:0/3400011971 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff2641558e0 0x7ff2640ae9b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2540097a0 tx=0x7ff25402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ea6d22919c618b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff261d9b640 1 -- 172.21.15.7:0/3400011971 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254037070 con 0x7ff2641558e0 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff261d9b640 1 -- 172.21.15.7:0/3400011971 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25402adb0 con 0x7ff2641558e0 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff261d9b640 1 -- 172.21.15.7:0/3400011971 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254032660 con 0x7ff2641558e0 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff269096640 1 -- 172.21.15.7:0/3400011971 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff2641558e0 msgr2=0x7ff2640ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff269096640 1 --2- 172.21.15.7:0/3400011971 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff2641558e0 0x7ff2640ae9b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2540097a0 tx=0x7ff25402a060 comp rx=0 tx=0).stop 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff269096640 1 -- 172.21.15.7:0/3400011971 shutdown_connections 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff269096640 1 --2- 172.21.15.7:0/3400011971 >> v2:172.21.15.18:3300/0 conn(0x7ff26415f600 0x7ff264072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff269096640 1 --2- 172.21.15.7:0/3400011971 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff2641558e0 0x7ff2640ae9b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff269096640 1 -- 172.21.15.7:0/3400011971 >> 172.21.15.7:0/3400011971 conn(0x7ff2640b1e60 msgr2=0x7ff2640b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:46.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff269096640 1 -- 172.21.15.7:0/3400011971 shutdown_connections 2024-09-15T23:37:46.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.308+0000 7ff269096640 1 -- 172.21.15.7:0/3400011971 wait complete. 2024-09-15T23:37:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.310+0000 7ff269096640 1 Processor -- start 2024-09-15T23:37:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.310+0000 7ff269096640 1 -- start start 2024-09-15T23:37:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.310+0000 7ff269096640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff26406e060 0x7ff26414f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.310+0000 7ff269096640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2641558e0 0x7ff264154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.310+0000 7ff269096640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff26415f600 0x7ff2641f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.310+0000 7ff269096640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff264155cc0 con 0x7ff2641558e0 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.310+0000 7ff269096640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff2640aeef0 con 0x7ff26415f600 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.310+0000 7ff269096640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7ff264155e40 con 0x7ff26406e060 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2641558e0 0x7ff264154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2641558e0 0x7ff264154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38698/0 (socket says 172.21.15.7:38698) 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26259c640 1 -- 172.21.15.7:0/651147815 learned_addr learned my addr 172.21.15.7:0/651147815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26359e640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff26415f600 0x7ff2641f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff262d9d640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff26406e060 0x7ff26414f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26259c640 1 -- 172.21.15.7:0/651147815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff26406e060 msgr2=0x7ff26414f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26259c640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff26406e060 0x7ff26414f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26259c640 1 -- 172.21.15.7:0/651147815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff26415f600 msgr2=0x7ff2641f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26259c640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff26415f600 0x7ff2641f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26259c640 1 -- 172.21.15.7:0/651147815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2641550f0 con 0x7ff2641558e0 2024-09-15T23:37:46.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff26259c640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2641558e0 0x7ff264154bb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c500 tx=0x7ff25800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff24b7fe640 1 -- 172.21.15.7:0/651147815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258018070 con 0x7ff2641558e0 2024-09-15T23:37:46.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff24b7fe640 1 -- 172.21.15.7:0/651147815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff258007580 con 0x7ff2641558e0 2024-09-15T23:37:46.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff24b7fe640 1 -- 172.21.15.7:0/651147815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258004a20 con 0x7ff2641558e0 2024-09-15T23:37:46.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff264153e00 con 0x7ff2641558e0 2024-09-15T23:37:46.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.311+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641543c0 con 0x7ff2641558e0 2024-09-15T23:37:46.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.312+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff2640ae9b0 con 0x7ff2641558e0 2024-09-15T23:37:46.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.313+0000 7ff24b7fe640 1 -- 172.21.15.7:0/651147815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7ff25801b070 con 0x7ff2641558e0 2024-09-15T23:37:46.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.313+0000 7ff24b7fe640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff25006c2e0 0x7ff25006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.313+0000 7ff24b7fe640 1 -- 172.21.15.7:0/651147815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7ff25808a760 con 0x7ff2641558e0 2024-09-15T23:37:46.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.313+0000 7ff262d9d640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff25006c2e0 0x7ff25006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.313+0000 7ff24b7fe640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff250072cd0 0x7ff250075110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.313+0000 7ff24b7fe640 1 -- 172.21.15.7:0/651147815 --> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2500757e0 con 0x7ff250072cd0 2024-09-15T23:37:46.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.313+0000 7ff24b7fe640 1 -- 172.21.15.7:0/651147815 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff258091050 con 0x7ff2641558e0 2024-09-15T23:37:46.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.314+0000 7ff26359e640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff250072cd0 0x7ff250075110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.314+0000 7ff262d9d640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff25006c2e0 0x7ff25006e7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff254034000 tx=0x7ff25402e000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.314+0000 7ff26359e640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff250072cd0 0x7ff250075110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.315+0000 7ff24b7fe640 1 -- 172.21.15.7:0/651147815 <== osd.3 v2:172.21.15.7:6801/2160174572 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff2500757e0 con 0x7ff250072cd0 2024-09-15T23:37:46.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.327+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 --> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff26409e1e0 con 0x7ff250072cd0 2024-09-15T23:37:46.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff24b7fe640 1 -- 172.21.15.7:0/651147815 <== osd.3 v2:172.21.15.7:6801/2160174572 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff26409e1e0 con 0x7ff250072cd0 2024-09-15T23:37:46.329 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T23:37:46.329 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff250072cd0 msgr2=0x7ff250075110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6801/2160174572,v1:172.21.15.7:6803/2160174572] conn(0x7ff250072cd0 0x7ff250075110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff25006c2e0 msgr2=0x7ff25006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7ff25006c2e0 0x7ff25006e7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff254034000 tx=0x7ff25402e000 comp rx=0 tx=0).stop 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2641558e0 msgr2=0x7ff264154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 --2- 172.21.15.7:0/651147815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2641558e0 0x7ff264154bb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c500 tx=0x7ff25800c9c0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff26359e640 1 -- 172.21.15.7:0/651147815 reap_dead start 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 shutdown_connections 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 >> 172.21.15.7:0/651147815 conn(0x7ff2640b1e60 msgr2=0x7ff26415dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:46.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 shutdown_connections 2024-09-15T23:37:46.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.328+0000 7ff269096640 1 -- 172.21.15.7:0/651147815 wait complete. 2024-09-15T23:37:46.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T23:37:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.441+0000 7f4dbaf81640 1 Processor -- start 2024-09-15T23:37:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.441+0000 7f4dbaf81640 1 -- start start 2024-09-15T23:37:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4dbaf81640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4db409aa40 0x7f4db409ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4dbaf81640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db4160120 0x7f4db40d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4dbaf81640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db409b360 con 0x7f4db40d0c10 2024-09-15T23:37:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4dbaf81640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4db4070300 con 0x7f4db409aa40 2024-09-15T23:37:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4dbaf81640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f4db40d1120 con 0x7f4db4160120 2024-09-15T23:37:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4db8d1d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4db409aa40 0x7f4db409ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4db8d1d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4db409aa40 0x7f4db409ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38710/0 (socket says 172.21.15.7:38710) 2024-09-15T23:37:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4db8d1d640 1 -- 172.21.15.7:0/1236956420 learned_addr learned my addr 172.21.15.7:0/1236956420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4db3fff640 1 --2- 172.21.15.7:0/1236956420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db4160120 0x7f4db40d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4db8d1d640 1 -- 172.21.15.7:0/1236956420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db4160120 msgr2=0x7f4db40d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4db8d1d640 1 --2- 172.21.15.7:0/1236956420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db4160120 0x7f4db40d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4db8d1d640 1 -- 172.21.15.7:0/1236956420 >> v1:172.21.15.18:6789/0 conn(0x7f4db40d0c10 legacy=0x7f4db4070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4db8d1d640 1 -- 172.21.15.7:0/1236956420 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db40d1400 con 0x7f4db409aa40 2024-09-15T23:37:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.442+0000 7f4db3fff640 1 --2- 172.21.15.7:0/1236956420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db4160120 0x7f4db40d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.443+0000 7f4db8d1d640 1 --2- 172.21.15.7:0/1236956420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db409aa40 0x7f4db409ae20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009870 tx=0x7f4da802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb29a6546a4ab08 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.443+0000 7f4db37fe640 1 -- 172.21.15.7:0/1236956420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8037070 con 0x7f4db409aa40 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.443+0000 7f4db37fe640 1 -- 172.21.15.7:0/1236956420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da8032440 con 0x7f4db409aa40 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.443+0000 7f4db37fe640 1 -- 172.21.15.7:0/1236956420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8032810 con 0x7f4db409aa40 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.444+0000 7f4dbaf81640 1 -- 172.21.15.7:0/1236956420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db409aa40 msgr2=0x7f4db409ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.444+0000 7f4dbaf81640 1 --2- 172.21.15.7:0/1236956420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db409aa40 0x7f4db409ae20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009870 tx=0x7f4da802a270 comp rx=0 tx=0).stop 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.444+0000 7f4dbaf81640 1 -- 172.21.15.7:0/1236956420 shutdown_connections 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.444+0000 7f4dbaf81640 1 --2- 172.21.15.7:0/1236956420 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db4160120 0x7f4db40d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.444+0000 7f4dbaf81640 1 --2- 172.21.15.7:0/1236956420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db409aa40 0x7f4db409ae20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.444+0000 7f4dbaf81640 1 -- 172.21.15.7:0/1236956420 >> 172.21.15.7:0/1236956420 conn(0x7f4db40d2420 msgr2=0x7f4db40d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.444+0000 7f4dbaf81640 1 -- 172.21.15.7:0/1236956420 shutdown_connections 2024-09-15T23:37:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.444+0000 7f4dbaf81640 1 -- 172.21.15.7:0/1236956420 wait complete. 2024-09-15T23:37:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.445+0000 7f4dbaf81640 1 Processor -- start 2024-09-15T23:37:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.445+0000 7f4dbaf81640 1 -- start start 2024-09-15T23:37:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4dbaf81640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4db409aa40 0x7f4db41e8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4dbaf81640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db40d0c10 0x7f4db414f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4dbaf81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db4160120 0x7f4db40b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4dbaf81640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4161f70 con 0x7f4db4160120 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4dbaf81640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4db40d19c0 con 0x7f4db409aa40 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4dbaf81640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f4db41620f0 con 0x7f4db40d0c10 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4db951e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db4160120 0x7f4db40b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4db951e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db4160120 0x7f4db40b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38720/0 (socket says 172.21.15.7:38720) 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4db951e640 1 -- 172.21.15.7:0/3788384927 learned_addr learned my addr 172.21.15.7:0/3788384927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4db8d1d640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4db409aa40 0x7f4db41e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.446+0000 7f4db3fff640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db40d0c10 0x7f4db414f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4db951e640 1 -- 172.21.15.7:0/3788384927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db40d0c10 msgr2=0x7f4db414f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4db951e640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db40d0c10 0x7f4db414f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4db951e640 1 -- 172.21.15.7:0/3788384927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4db409aa40 msgr2=0x7f4db41e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4db951e640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4db409aa40 0x7f4db41e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4db951e640 1 -- 172.21.15.7:0/3788384927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db40b6960 con 0x7f4db4160120 2024-09-15T23:37:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4db951e640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db4160120 0x7f4db40b0510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00c6b0 tx=0x7f4dac00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4db17fa640 1 -- 172.21.15.7:0/3788384927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac003b70 con 0x7f4db4160120 2024-09-15T23:37:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4db17fa640 1 -- 172.21.15.7:0/3788384927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dac003d30 con 0x7f4db4160120 2024-09-15T23:37:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db40b6c40 con 0x7f4db4160120 2024-09-15T23:37:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4db17fa640 1 -- 172.21.15.7:0/3788384927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac005310 con 0x7f4db4160120 2024-09-15T23:37:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.447+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db40b7200 con 0x7f4db4160120 2024-09-15T23:37:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.448+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db40b7400 con 0x7f4db4160120 2024-09-15T23:37:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.448+0000 7f4db17fa640 1 -- 172.21.15.7:0/3788384927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f4dac0054b0 con 0x7f4db4160120 2024-09-15T23:37:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.449+0000 7f4db17fa640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f4d9806c2b0 0x7f4d9806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.449+0000 7f4db8d1d640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f4d9806c2b0 0x7f4d9806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.449+0000 7f4db17fa640 1 -- 172.21.15.7:0/3788384927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f4dac08b230 con 0x7f4db4160120 2024-09-15T23:37:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.450+0000 7f4db8d1d640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f4d9806c2b0 0x7f4d9806e770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4da8034000 tx=0x7f4da802e000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.453+0000 7f4db17fa640 1 -- 172.21.15.7:0/3788384927 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4db40b7400 con 0x7f4db4160120 2024-09-15T23:37:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.627+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f4db41e8a70 con 0x7f4d9806c2b0 2024-09-15T23:37:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.627+0000 7f4db17fa640 1 -- 172.21.15.7:0/3788384927 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4da80326f0 con 0x7f4d9806c2b0 2024-09-15T23:37:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T23:37:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f4d9806c2b0 msgr2=0x7f4d9806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f4d9806c2b0 0x7f4d9806e770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4da8034000 tx=0x7f4da802e000 comp rx=0 tx=0).stop 2024-09-15T23:37:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db4160120 msgr2=0x7f4db40b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db4160120 0x7f4db40b0510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00c6b0 tx=0x7f4dac00cb70 comp rx=0 tx=0).stop 2024-09-15T23:37:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 shutdown_connections 2024-09-15T23:37:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f4d9806c2b0 0x7f4d9806e770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4db4160120 0x7f4db40b0510 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4db40d0c10 0x7f4db414f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 --2- 172.21.15.7:0/3788384927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4db409aa40 0x7f4db41e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 >> 172.21.15.7:0/3788384927 conn(0x7f4db40d2420 msgr2=0x7f4db40cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 shutdown_connections 2024-09-15T23:37:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.630+0000 7f4dbaf81640 1 -- 172.21.15.7:0/3788384927 wait complete. 2024-09-15T23:37:46.642 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T23:37:46.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T23:37:46.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.748+0000 7fdc3f6c0640 1 Processor -- start 2024-09-15T23:37:46.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.748+0000 7fdc3f6c0640 1 -- start start 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc3f6c0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc38070ea0 0x7fdc380d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc3f6c0640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc3809bae0 con 0x7fdc3815e850 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc3f6c0640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc380d07e0 con 0x7fdc3809b450 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc3f6c0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fdc380d05b0 con 0x7fdc38070ea0 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc3d45c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdc3815e850 0x7fdc38070790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57654/0 (socket says 172.21.15.7:57654) 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc3d45c640 1 -- 172.21.15.7:0/892133184 learned_addr learned my addr 172.21.15.7:0/892133184 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc3cc5b640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc38070ea0 0x7fdc380d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1799346107 0 0) 0x7fdc3809bae0 con 0x7fdc3815e850 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc24003520 con 0x7fdc3815e850 2024-09-15T23:37:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1595431035 0 0) 0x7fdc24003520 con 0x7fdc3815e850 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc38070ea0 msgr2=0x7fdc380d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc38070ea0 0x7fdc380d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 >> v1:172.21.15.18:6789/0 conn(0x7fdc3809b450 legacy=0x7fdc3806cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc380d0ac0 con 0x7fdc3815e850 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7fdc34001670 con 0x7fdc3815e850 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 >> v1:172.21.15.7:6789/0 conn(0x7fdc3815e850 legacy=0x7fdc38070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc240039e0 0x7fdc24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc38070ea0 0x7fdc24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24004d40 0x7fdc240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2400b9c0 con 0x7fdc24004d40 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2400bb70 con 0x7fdc240039e0 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.749+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2400bd70 con 0x7fdc38070ea0 2024-09-15T23:37:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.750+0000 7fdc3d45c640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc240039e0 0x7fdc24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.750+0000 7fdc3dc5d640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24004d40 0x7fdc240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.750+0000 7fdc3cc5b640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc38070ea0 0x7fdc24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.750+0000 7fdc3dc5d640 1 -- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc38070ea0 msgr2=0x7fdc24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.750+0000 7fdc3dc5d640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc38070ea0 0x7fdc24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.750+0000 7fdc3dc5d640 1 -- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc240039e0 msgr2=0x7fdc24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.750+0000 7fdc3dc5d640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc240039e0 0x7fdc24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.750+0000 7fdc3dc5d640 1 -- 172.21.15.7:0/892133184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc24003520 con 0x7fdc24004d40 2024-09-15T23:37:46.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc3dc5d640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24004d40 0x7fdc240051c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdc30001f60 tx=0x7fdc24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3578a74ab27c76d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30001b30 con 0x7fdc24004d40 2024-09-15T23:37:46.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc30031620 con 0x7fdc24004d40 2024-09-15T23:37:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc2ffff640 1 -- 172.21.15.7:0/892133184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30031a10 con 0x7fdc24004d40 2024-09-15T23:37:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/892133184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24004d40 msgr2=0x7fdc240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc3f6c0640 1 --2- 172.21.15.7:0/892133184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24004d40 0x7fdc240051c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdc30001f60 tx=0x7fdc24005830 comp rx=0 tx=0).stop 2024-09-15T23:37:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc3dc5d640 1 -- 172.21.15.7:0/892133184 reap_dead start 2024-09-15T23:37:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/892133184 shutdown_connections 2024-09-15T23:37:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/892133184 >> 172.21.15.7:0/892133184 conn(0x7fdc380d2420 msgr2=0x7fdc380d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/892133184 shutdown_connections 2024-09-15T23:37:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.751+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/892133184 wait complete. 2024-09-15T23:37:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.752+0000 7fdc3f6c0640 1 Processor -- start 2024-09-15T23:37:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3f6c0640 1 -- start start 2024-09-15T23:37:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3f6c0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc381f4410 0x7fdc3806de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3f6c0640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc381f47f0 0x7fdc3806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3f6c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc380b7620 0x7fdc380b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3f6c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3815ec30 con 0x7fdc380b7620 2024-09-15T23:37:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3f6c0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdc380d1110 con 0x7fdc381f4410 2024-09-15T23:37:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3f6c0640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fdc380d15a0 con 0x7fdc381f47f0 2024-09-15T23:37:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3dc5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc380b7620 0x7fdc380b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3dc5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc380b7620 0x7fdc380b7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38746/0 (socket says 172.21.15.7:38746) 2024-09-15T23:37:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3dc5d640 1 -- 172.21.15.7:0/1829428195 learned_addr learned my addr 172.21.15.7:0/1829428195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3d45c640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc381f4410 0x7fdc3806de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.753+0000 7fdc3cc5b640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc381f47f0 0x7fdc3806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc3dc5d640 1 -- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc381f47f0 msgr2=0x7fdc3806e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc3dc5d640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdc381f47f0 0x7fdc3806e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc3dc5d640 1 -- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc381f4410 msgr2=0x7fdc3806de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc3dc5d640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc381f4410 0x7fdc3806de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc3dc5d640 1 -- 172.21.15.7:0/1829428195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc380b53a0 con 0x7fdc380b7620 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc3dc5d640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc380b7620 0x7fdc380b7ab0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdc3002aa00 tx=0x7fdc300015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc2dffb640 1 -- 172.21.15.7:0/1829428195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30001b30 con 0x7fdc380b7620 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc2dffb640 1 -- 172.21.15.7:0/1829428195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc30006d60 con 0x7fdc380b7620 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc380b5680 con 0x7fdc380b7620 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc2dffb640 1 -- 172.21.15.7:0/1829428195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30031a10 con 0x7fdc380b7620 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.754+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc380b5c40 con 0x7fdc380b7620 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.755+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc380b3ce0 con 0x7fdc380b7620 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.756+0000 7fdc2dffb640 1 -- 172.21.15.7:0/1829428195 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fdc30044020 con 0x7fdc380b7620 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.756+0000 7fdc2dffb640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdc1c06c2b0 0x7fdc1c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.756+0000 7fdc2dffb640 1 -- 172.21.15.7:0/1829428195 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fdc300ab730 con 0x7fdc380b7620 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.756+0000 7fdc3d45c640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdc1c06c2b0 0x7fdc1c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.757+0000 7fdc2dffb640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7fdc1c072ca0 0x7fdc1c0750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.757+0000 7fdc2dffb640 1 -- 172.21.15.7:0/1829428195 --> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc30009eb0 con 0x7fdc1c072ca0 2024-09-15T23:37:46.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.757+0000 7fdc2dffb640 1 -- 172.21.15.7:0/1829428195 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdc300b2050 con 0x7fdc380b7620 2024-09-15T23:37:46.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.757+0000 7fdc3cc5b640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7fdc1c072ca0 0x7fdc1c0750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.757+0000 7fdc3d45c640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdc1c06c2b0 0x7fdc1c06e770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc34007d90 tx=0x7fdc34000bc0 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.757+0000 7fdc3cc5b640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7fdc1c072ca0 0x7fdc1c0750e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.758+0000 7fdc2dffb640 1 -- 172.21.15.7:0/1829428195 <== osd.4 v2:172.21.15.18:6800/2251983885 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc30009eb0 con 0x7fdc1c072ca0 2024-09-15T23:37:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.771+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 --> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdc380d1930 con 0x7fdc1c072ca0 2024-09-15T23:37:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.771+0000 7fdc2dffb640 1 -- 172.21.15.7:0/1829428195 <== osd.4 v2:172.21.15.18:6800/2251983885 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdc380d1930 con 0x7fdc1c072ca0 2024-09-15T23:37:46.772 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T23:37:46.772 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:46.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.771+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7fdc1c072ca0 msgr2=0x7fdc1c0750e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.771+0000 7fdc3f6c0640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.18:6800/2251983885,v1:172.21.15.18:6801/2251983885] conn(0x7fdc1c072ca0 0x7fdc1c0750e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.771+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdc1c06c2b0 msgr2=0x7fdc1c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.771+0000 7fdc3f6c0640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fdc1c06c2b0 0x7fdc1c06e770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc34007d90 tx=0x7fdc34000bc0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.771+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc380b7620 msgr2=0x7fdc380b7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.771+0000 7fdc3f6c0640 1 --2- 172.21.15.7:0/1829428195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc380b7620 0x7fdc380b7ab0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdc3002aa00 tx=0x7fdc300015b0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.771+0000 7fdc3dc5d640 1 -- 172.21.15.7:0/1829428195 reap_dead start 2024-09-15T23:37:46.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.772+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 shutdown_connections 2024-09-15T23:37:46.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.772+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 >> 172.21.15.7:0/1829428195 conn(0x7fdc380d2420 msgr2=0x7fdc380b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:46.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.772+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 shutdown_connections 2024-09-15T23:37:46.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.772+0000 7fdc3f6c0640 1 -- 172.21.15.7:0/1829428195 wait complete. 2024-09-15T23:37:46.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T23:37:46.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.885+0000 7fbadff9f640 1 Processor -- start 2024-09-15T23:37:46.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.886+0000 7fbadff9f640 1 -- start start 2024-09-15T23:37:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.886+0000 7fbadff9f640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbad815e990 0x7fbad80d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.886+0000 7fbadff9f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad8163170 con 0x7fbad80d24e0 2024-09-15T23:37:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.886+0000 7fbadff9f640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad80d2980 con 0x7fbad815e5b0 2024-09-15T23:37:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.886+0000 7fbadff9f640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fbad80d2c40 con 0x7fbad815e990 2024-09-15T23:37:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadd53a640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbad80d24e0 0x7fbad8162a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57670/0 (socket says 172.21.15.7:57670) 2024-09-15T23:37:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadd53a640 1 -- 172.21.15.7:0/602561848 learned_addr learned my addr 172.21.15.7:0/602561848 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2068208341 0 0) 0x7fbad8163170 con 0x7fbad80d24e0 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbac4003520 con 0x7fbad80d24e0 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbade53c640 1 --2- 172.21.15.7:0/602561848 >> v2:172.21.15.18:3300/0 conn(0x7fbad815e990 0x7fbad80d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 530957469 0 0) 0x7fbac4003520 con 0x7fbad80d24e0 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 >> v2:172.21.15.18:3300/0 conn(0x7fbad815e990 msgr2=0x7fbad80d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 --2- 172.21.15.7:0/602561848 >> v2:172.21.15.18:3300/0 conn(0x7fbad815e990 0x7fbad80d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 >> v1:172.21.15.18:6789/0 conn(0x7fbad815e5b0 legacy=0x7fbad8162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad80d2f20 con 0x7fbad80d24e0 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7fbac80017f0 con 0x7fbad80d24e0 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 >> v1:172.21.15.7:6789/0 conn(0x7fbad80d24e0 legacy=0x7fbad8162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbac40039b0 0x7fbac4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac4004800 0x7fbac4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbac40051c0 0x7fbac400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbad80d2c40 con 0x7fbac4004800 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbac400c060 con 0x7fbac40039b0 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbac400c260 con 0x7fbac40051c0 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbadd53a640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac4004800 0x7fbac4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbaddd3b640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbac40039b0 0x7fbac4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.887+0000 7fbade53c640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbac40051c0 0x7fbac400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbadd53a640 1 -- 172.21.15.7:0/602561848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbac40051c0 msgr2=0x7fbac400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbadd53a640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbac40051c0 0x7fbac400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbadd53a640 1 -- 172.21.15.7:0/602561848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbac40039b0 msgr2=0x7fbac4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbadd53a640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbac40039b0 0x7fbac4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbadd53a640 1 -- 172.21.15.7:0/602561848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac4003520 con 0x7fbac4004800 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbade53c640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbac40051c0 0x7fbac400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbade53c640 1 -- 172.21.15.7:0/602561848 reap_dead start 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbadd53a640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac4004800 0x7fbac4004c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c240 tx=0x7fbac4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08c13553668b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8007ce0 con 0x7fbac4004800 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac8007270 con 0x7fbac4004800 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.888+0000 7fbadcd39640 1 -- 172.21.15.7:0/602561848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac80057a0 con 0x7fbac4004800 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.889+0000 7fbadff9f640 1 -- 172.21.15.7:0/602561848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac4004800 msgr2=0x7fbac4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.889+0000 7fbadff9f640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac4004800 0x7fbac4004c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c240 tx=0x7fbac4005830 comp rx=0 tx=0).stop 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.889+0000 7fbadff9f640 1 -- 172.21.15.7:0/602561848 shutdown_connections 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.889+0000 7fbadff9f640 1 --2- 172.21.15.7:0/602561848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac4004800 0x7fbac4004c80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.889+0000 7fbadff9f640 1 -- 172.21.15.7:0/602561848 >> 172.21.15.7:0/602561848 conn(0x7fbad808bfd0 msgr2=0x7fbad80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.889+0000 7fbadff9f640 1 -- 172.21.15.7:0/602561848 shutdown_connections 2024-09-15T23:37:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.889+0000 7fbadff9f640 1 -- 172.21.15.7:0/602561848 wait complete. 2024-09-15T23:37:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.890+0000 7fbadff9f640 1 Processor -- start 2024-09-15T23:37:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.890+0000 7fbadff9f640 1 -- start start 2024-09-15T23:37:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadff9f640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbac4004800 0x7fbad81f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadff9f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad815a9f0 0x7fbad80b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadff9f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbad80b90a0 0x7fbad815b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadff9f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbad80b39c0 con 0x7fbad815a9f0 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadff9f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbad80d34e0 con 0x7fbad80b90a0 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadff9f640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbad80b3b40 con 0x7fbac4004800 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadd53a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad815a9f0 0x7fbad80b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadd53a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad815a9f0 0x7fbad80b8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51642/0 (socket says 172.21.15.7:51642) 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadd53a640 1 -- 172.21.15.7:0/3842415170 learned_addr learned my addr 172.21.15.7:0/3842415170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadd53a640 1 -- 172.21.15.7:0/3842415170 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbac4004800 msgr2=0x7fbad81f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadd53a640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbac4004800 0x7fbad81f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadd53a640 1 -- 172.21.15.7:0/3842415170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbad80b90a0 msgr2=0x7fbad815b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadd53a640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbad80b90a0 0x7fbad815b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.891+0000 7fbadd53a640 1 -- 172.21.15.7:0/3842415170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad80b5b20 con 0x7fbad815a9f0 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.892+0000 7fbadd53a640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad815a9f0 0x7fbad80b8b60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c360 tx=0x7fbad80d30f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.892+0000 7fbace7fc640 1 -- 172.21.15.7:0/3842415170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac803f070 con 0x7fbad815a9f0 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.892+0000 7fbace7fc640 1 -- 172.21.15.7:0/3842415170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac80015f0 con 0x7fbad815a9f0 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.892+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad80b99e0 con 0x7fbad815a9f0 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.892+0000 7fbace7fc640 1 -- 172.21.15.7:0/3842415170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8007420 con 0x7fbad815a9f0 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.892+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad80b9520 con 0x7fbad815a9f0 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.893+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad8162350 con 0x7fbad815a9f0 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.894+0000 7fbace7fc640 1 -- 172.21.15.7:0/3842415170 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fbac8042070 con 0x7fbad815a9f0 2024-09-15T23:37:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.894+0000 7fbace7fc640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbabc06c2b0 0x7fbabc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:46.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.894+0000 7fbace7fc640 1 -- 172.21.15.7:0/3842415170 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fbac80aab60 con 0x7fbad815a9f0 2024-09-15T23:37:46.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.894+0000 7fbaddd3b640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbabc06c2b0 0x7fbabc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:46.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.895+0000 7fbaddd3b640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbabc06c2b0 0x7fbabc06e770 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbad4000c00 tx=0x7fbad4009040 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:46.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:46.898+0000 7fbace7fc640 1 -- 172.21.15.7:0/3842415170 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbac8049070 con 0x7fbad815a9f0 2024-09-15T23:37:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.073+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fbad80ba560 con 0x7fbabc06c2b0 2024-09-15T23:37:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.074+0000 7fbace7fc640 1 -- 172.21.15.7:0/3842415170 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbad80ba560 con 0x7fbabc06c2b0 2024-09-15T23:37:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T23:37:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbabc06c2b0 msgr2=0x7fbabc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbabc06c2b0 0x7fbabc06e770 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbad4000c00 tx=0x7fbad4009040 comp rx=0 tx=0).stop 2024-09-15T23:37:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad815a9f0 msgr2=0x7fbad80b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad815a9f0 0x7fbad80b8b60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c360 tx=0x7fbad80d30f0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 shutdown_connections 2024-09-15T23:37:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbabc06c2b0 0x7fbabc06e770 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbad80b90a0 0x7fbad815b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad815a9f0 0x7fbad80b8b60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 --2- 172.21.15.7:0/3842415170 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbac4004800 0x7fbad81f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 >> 172.21.15.7:0/3842415170 conn(0x7fbad808bfd0 msgr2=0x7fbad806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 shutdown_connections 2024-09-15T23:37:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.076+0000 7fbadff9f640 1 -- 172.21.15.7:0/3842415170 wait complete. 2024-09-15T23:37:47.088 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T23:37:47.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T23:37:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.194+0000 7f1c39ae4640 1 Processor -- start 2024-09-15T23:37:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.194+0000 7f1c39ae4640 1 -- start start 2024-09-15T23:37:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.195+0000 7f1c39ae4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c341558e0 0x7f1c340ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.195+0000 7f1c39ae4640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c340af270 con 0x7f1c3406e060 2024-09-15T23:37:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.195+0000 7f1c39ae4640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c34072020 con 0x7f1c3415f600 2024-09-15T23:37:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.195+0000 7f1c39ae4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f1c340b3160 con 0x7f1c341558e0 2024-09-15T23:37:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.195+0000 7f1c32ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1c3406e060 0x7f1c3415eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38638/0 (socket says 172.21.15.7:38638) 2024-09-15T23:37:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.196+0000 7f1c32ffd640 1 -- 172.21.15.7:0/2455466413 learned_addr learned my addr 172.21.15.7:0/2455466413 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.196+0000 7f1c337fe640 1 --2- 172.21.15.7:0/2455466413 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c341558e0 0x7f1c340ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.196+0000 7f1c327fc640 1 -- 172.21.15.7:0/2455466413 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982052413 0 0) 0x7f1c340af270 con 0x7f1c3406e060 2024-09-15T23:37:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.196+0000 7f1c327fc640 1 -- 172.21.15.7:0/2455466413 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c1c003520 con 0x7f1c3406e060 2024-09-15T23:37:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.196+0000 7f1c327fc640 1 -- 172.21.15.7:0/2455466413 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3872019878 0 0) 0x7f1c34072020 con 0x7f1c3415f600 2024-09-15T23:37:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.196+0000 7f1c327fc640 1 -- 172.21.15.7:0/2455466413 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c340af270 con 0x7f1c3415f600 2024-09-15T23:37:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.196+0000 7f1c337fe640 1 -- 172.21.15.7:0/2455466413 >> v1:172.21.15.18:6789/0 conn(0x7f1c3415f600 legacy=0x7f1c34072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.196+0000 7f1c337fe640 1 -- 172.21.15.7:0/2455466413 >> v1:172.21.15.7:6789/0 conn(0x7f1c3406e060 legacy=0x7f1c3415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.196+0000 7f1c337fe640 1 -- 172.21.15.7:0/2455466413 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c34072300 con 0x7f1c341558e0 2024-09-15T23:37:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.197+0000 7f1c337fe640 1 --2- 172.21.15.7:0/2455466413 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c341558e0 0x7f1c340ae9b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c240097d0 tx=0x7f1c2402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1aa53ff845ef24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.197+0000 7f1c327fc640 1 -- 172.21.15.7:0/2455466413 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24037070 con 0x7f1c341558e0 2024-09-15T23:37:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.197+0000 7f1c327fc640 1 -- 172.21.15.7:0/2455466413 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2402ae50 con 0x7f1c341558e0 2024-09-15T23:37:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.197+0000 7f1c327fc640 1 -- 172.21.15.7:0/2455466413 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24032640 con 0x7f1c341558e0 2024-09-15T23:37:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.197+0000 7f1c39ae4640 1 -- 172.21.15.7:0/2455466413 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c341558e0 msgr2=0x7f1c340ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.197+0000 7f1c39ae4640 1 --2- 172.21.15.7:0/2455466413 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c341558e0 0x7f1c340ae9b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c240097d0 tx=0x7f1c2402a100 comp rx=0 tx=0).stop 2024-09-15T23:37:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.198+0000 7f1c39ae4640 1 -- 172.21.15.7:0/2455466413 shutdown_connections 2024-09-15T23:37:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.198+0000 7f1c39ae4640 1 --2- 172.21.15.7:0/2455466413 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c341558e0 0x7f1c340ae9b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.198+0000 7f1c39ae4640 1 -- 172.21.15.7:0/2455466413 >> 172.21.15.7:0/2455466413 conn(0x7f1c340b1e60 msgr2=0x7f1c340b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.198+0000 7f1c39ae4640 1 -- 172.21.15.7:0/2455466413 shutdown_connections 2024-09-15T23:37:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.198+0000 7f1c39ae4640 1 -- 172.21.15.7:0/2455466413 wait complete. 2024-09-15T23:37:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.199+0000 7f1c39ae4640 1 Processor -- start 2024-09-15T23:37:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.199+0000 7f1c39ae4640 1 -- start start 2024-09-15T23:37:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c39ae4640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c3406e060 0x7f1c340b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c39ae4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c341558e0 0x7f1c34151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c39ae4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c3415f600 0x7f1c34158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c39ae4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34155cc0 con 0x7f1c3415f600 2024-09-15T23:37:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c39ae4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1c340af550 con 0x7f1c341558e0 2024-09-15T23:37:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c39ae4640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f1c34155e40 con 0x7f1c3406e060 2024-09-15T23:37:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c33fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c3415f600 0x7f1c34158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c33fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c3415f600 0x7f1c34158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51658/0 (socket says 172.21.15.7:51658) 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c33fff640 1 -- 172.21.15.7:0/903811444 learned_addr learned my addr 172.21.15.7:0/903811444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c337fe640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c3406e060 0x7f1c340b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c32ffd640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c341558e0 0x7f1c34151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c33fff640 1 -- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c3406e060 msgr2=0x7f1c340b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c33fff640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c3406e060 0x7f1c340b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c33fff640 1 -- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c341558e0 msgr2=0x7f1c34151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c33fff640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c341558e0 0x7f1c34151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c33fff640 1 -- 172.21.15.7:0/903811444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c34151ec0 con 0x7f1c3415f600 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c33fff640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c3415f600 0x7f1c34158c50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00c700 tx=0x7f1c2c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c1bfff640 1 -- 172.21.15.7:0/903811444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c003b70 con 0x7f1c3415f600 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c1bfff640 1 -- 172.21.15.7:0/903811444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2c003d10 con 0x7f1c3415f600 2024-09-15T23:37:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c340b4560 con 0x7f1c3415f600 2024-09-15T23:37:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.200+0000 7f1c1bfff640 1 -- 172.21.15.7:0/903811444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0052d0 con 0x7f1c3415f600 2024-09-15T23:37:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.201+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c340b4b20 con 0x7f1c3415f600 2024-09-15T23:37:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.201+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c340af760 con 0x7f1c3415f600 2024-09-15T23:37:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.202+0000 7f1c1bfff640 1 -- 172.21.15.7:0/903811444 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f1c2c005470 con 0x7f1c3415f600 2024-09-15T23:37:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.202+0000 7f1c1bfff640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1c2006c2b0 0x7f1c2006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.203+0000 7f1c1bfff640 1 -- 172.21.15.7:0/903811444 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f1c2c08b1d0 con 0x7f1c3415f600 2024-09-15T23:37:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.203+0000 7f1c337fe640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1c2006c2b0 0x7f1c2006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.203+0000 7f1c1bfff640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7f1c20072ca0 0x7f1c200750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.203+0000 7f1c1bfff640 1 -- 172.21.15.7:0/903811444 --> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c200757b0 con 0x7f1c20072ca0 2024-09-15T23:37:47.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.203+0000 7f1c1bfff640 1 -- 172.21.15.7:0/903811444 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1c2c092050 con 0x7f1c3415f600 2024-09-15T23:37:47.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.203+0000 7f1c32ffd640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7f1c20072ca0 0x7f1c200750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.203+0000 7f1c337fe640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1c2006c2b0 0x7f1c2006e770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034000 tx=0x7f1c2402e000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.203+0000 7f1c32ffd640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7f1c20072ca0 0x7f1c200750e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.204+0000 7f1c1bfff640 1 -- 172.21.15.7:0/903811444 <== osd.5 v2:172.21.15.18:6808/2875348132 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c200757b0 con 0x7f1c20072ca0 2024-09-15T23:37:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.217+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 --> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1c340ae9b0 con 0x7f1c20072ca0 2024-09-15T23:37:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.217+0000 7f1c1bfff640 1 -- 172.21.15.7:0/903811444 <== osd.5 v2:172.21.15.18:6808/2875348132 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1c340ae9b0 con 0x7f1c20072ca0 2024-09-15T23:37:47.219 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:47.219 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T23:37:47.219 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7f1c20072ca0 msgr2=0x7f1c200750e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.18:6808/2875348132,v1:172.21.15.18:6809/2875348132] conn(0x7f1c20072ca0 0x7f1c200750e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1c2006c2b0 msgr2=0x7f1c2006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1c2006c2b0 0x7f1c2006e770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034000 tx=0x7f1c2402e000 comp rx=0 tx=0).stop 2024-09-15T23:37:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c3415f600 msgr2=0x7f1c34158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 --2- 172.21.15.7:0/903811444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c3415f600 0x7f1c34158c50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00c700 tx=0x7f1c2c00cbc0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c33fff640 1 -- 172.21.15.7:0/903811444 reap_dead start 2024-09-15T23:37:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 shutdown_connections 2024-09-15T23:37:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 >> 172.21.15.7:0/903811444 conn(0x7f1c340b1e60 msgr2=0x7f1c3406e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 shutdown_connections 2024-09-15T23:37:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.218+0000 7f1c39ae4640 1 -- 172.21.15.7:0/903811444 wait complete. 2024-09-15T23:37:47.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T23:37:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.333+0000 7fb794d66640 1 Processor -- start 2024-09-15T23:37:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.333+0000 7fb794d66640 1 -- start start 2024-09-15T23:37:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb794d66640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb79015e6f0 0x7fb7900d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb794d66640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb7900d4f40 0x7fb7900d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb794d66640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7900d2ef0 con 0x7fb7900d1cb0 2024-09-15T23:37:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb794d66640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb7900d31d0 con 0x7fb79015e6f0 2024-09-15T23:37:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb794d66640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb7900d34b0 con 0x7fb7900d4f40 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb78e59c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb79015e6f0 0x7fb7900d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb78e59c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb79015e6f0 0x7fb7900d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51674/0 (socket says 172.21.15.7:51674) 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb78e59c640 1 -- 172.21.15.7:0/169620543 learned_addr learned my addr 172.21.15.7:0/169620543 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb78dd9b640 1 --2- 172.21.15.7:0/169620543 >> v2:172.21.15.18:3300/0 conn(0x7fb7900d4f40 0x7fb7900d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb78e59c640 1 -- 172.21.15.7:0/169620543 >> v2:172.21.15.18:3300/0 conn(0x7fb7900d4f40 msgr2=0x7fb7900d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb78e59c640 1 --2- 172.21.15.7:0/169620543 >> v2:172.21.15.18:3300/0 conn(0x7fb7900d4f40 0x7fb7900d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb78e59c640 1 -- 172.21.15.7:0/169620543 >> v1:172.21.15.18:6789/0 conn(0x7fb7900d1cb0 legacy=0x7fb7900cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb78e59c640 1 -- 172.21.15.7:0/169620543 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79006ce60 con 0x7fb79015e6f0 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.334+0000 7fb78dd9b640 1 --2- 172.21.15.7:0/169620543 >> v2:172.21.15.18:3300/0 conn(0x7fb7900d4f40 0x7fb7900d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.335+0000 7fb78e59c640 1 --2- 172.21.15.7:0/169620543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 0x7fb7900d0e60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb7900d5700 tx=0x7fb78002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca74ae39bae52947 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.335+0000 7fb78d59a640 1 -- 172.21.15.7:0/169620543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780037070 con 0x7fb79015e6f0 2024-09-15T23:37:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.335+0000 7fb78d59a640 1 -- 172.21.15.7:0/169620543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780032440 con 0x7fb79015e6f0 2024-09-15T23:37:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.335+0000 7fb78d59a640 1 -- 172.21.15.7:0/169620543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780032810 con 0x7fb79015e6f0 2024-09-15T23:37:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.336+0000 7fb794d66640 1 -- 172.21.15.7:0/169620543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 msgr2=0x7fb7900d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.336+0000 7fb794d66640 1 --2- 172.21.15.7:0/169620543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 0x7fb7900d0e60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb7900d5700 tx=0x7fb78002a270 comp rx=0 tx=0).stop 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.336+0000 7fb794d66640 1 -- 172.21.15.7:0/169620543 shutdown_connections 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.336+0000 7fb794d66640 1 --2- 172.21.15.7:0/169620543 >> v2:172.21.15.18:3300/0 conn(0x7fb7900d4f40 0x7fb7900d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.336+0000 7fb794d66640 1 --2- 172.21.15.7:0/169620543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 0x7fb7900d0e60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.336+0000 7fb794d66640 1 -- 172.21.15.7:0/169620543 >> 172.21.15.7:0/169620543 conn(0x7fb79008bfd0 msgr2=0x7fb7900d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.336+0000 7fb794d66640 1 -- 172.21.15.7:0/169620543 shutdown_connections 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.336+0000 7fb794d66640 1 -- 172.21.15.7:0/169620543 wait complete. 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.337+0000 7fb794d66640 1 Processor -- start 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.337+0000 7fb794d66640 1 -- start start 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb794d66640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7900d1cb0 0x7fb79015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb794d66640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb7900d4f40 0x7fb79015a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb794d66640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 0x7fb7900cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb794d66640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb79015ead0 con 0x7fb79015e6f0 2024-09-15T23:37:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb794d66640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb7900d2be0 con 0x7fb7900d1cb0 2024-09-15T23:37:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb794d66640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fb79015ec50 con 0x7fb7900d4f40 2024-09-15T23:37:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb78ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 0x7fb7900cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb78ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 0x7fb7900cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51680/0 (socket says 172.21.15.7:51680) 2024-09-15T23:37:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb78ed9d640 1 -- 172.21.15.7:0/1048741401 learned_addr learned my addr 172.21.15.7:0/1048741401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb78e59c640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7900d1cb0 0x7fb79015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb78dd9b640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb7900d4f40 0x7fb79015a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb78ed9d640 1 -- 172.21.15.7:0/1048741401 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb7900d4f40 msgr2=0x7fb79015a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb78ed9d640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb7900d4f40 0x7fb79015a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb78ed9d640 1 -- 172.21.15.7:0/1048741401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7900d1cb0 msgr2=0x7fb79015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.338+0000 7fb78ed9d640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7900d1cb0 0x7fb79015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.339+0000 7fb78ed9d640 1 -- 172.21.15.7:0/1048741401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79015ad80 con 0x7fb79015e6f0 2024-09-15T23:37:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.339+0000 7fb78dd9b640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb7900d4f40 0x7fb79015a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.339+0000 7fb78ed9d640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 0x7fb7900cb6d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb78800c7d0 tx=0x7fb78800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.339+0000 7fb77effd640 1 -- 172.21.15.7:0/1048741401 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788003b70 con 0x7fb79015e6f0 2024-09-15T23:37:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.339+0000 7fb77effd640 1 -- 172.21.15.7:0/1048741401 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb788003d30 con 0x7fb79015e6f0 2024-09-15T23:37:47.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.339+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7900ba330 con 0x7fb79015e6f0 2024-09-15T23:37:47.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.339+0000 7fb77effd640 1 -- 172.21.15.7:0/1048741401 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78800aa90 con 0x7fb79015e6f0 2024-09-15T23:37:47.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.339+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7900ce270 con 0x7fb79015e6f0 2024-09-15T23:37:47.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.340+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7900d0e60 con 0x7fb79015e6f0 2024-09-15T23:37:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.340+0000 7fb77effd640 1 -- 172.21.15.7:0/1048741401 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fb78800ac30 con 0x7fb79015e6f0 2024-09-15T23:37:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.341+0000 7fb77effd640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb77006c2b0 0x7fb77006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.341+0000 7fb77effd640 1 -- 172.21.15.7:0/1048741401 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fb78808b330 con 0x7fb79015e6f0 2024-09-15T23:37:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.341+0000 7fb78e59c640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb77006c2b0 0x7fb77006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.341+0000 7fb78e59c640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb77006c2b0 0x7fb77006e770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb780034000 tx=0x7fb78002e000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.345+0000 7fb77effd640 1 -- 172.21.15.7:0/1048741401 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb788059a40 con 0x7fb79015e6f0 2024-09-15T23:37:47.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.520+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fb7900cdd60 con 0x7fb77006c2b0 2024-09-15T23:37:47.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.520+0000 7fb77effd640 1 -- 172.21.15.7:0/1048741401 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb7900cdd60 con 0x7fb77006c2b0 2024-09-15T23:37:47.521 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T23:37:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb77006c2b0 msgr2=0x7fb77006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb77006c2b0 0x7fb77006e770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb780034000 tx=0x7fb78002e000 comp rx=0 tx=0).stop 2024-09-15T23:37:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 msgr2=0x7fb7900cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 0x7fb7900cb6d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb78800c7d0 tx=0x7fb78800cc90 comp rx=0 tx=0).stop 2024-09-15T23:37:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 shutdown_connections 2024-09-15T23:37:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fb77006c2b0 0x7fb77006e770 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79015e6f0 0x7fb7900cb6d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb7900d4f40 0x7fb79015a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 --2- 172.21.15.7:0/1048741401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7900d1cb0 0x7fb79015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 >> 172.21.15.7:0/1048741401 conn(0x7fb79008bfd0 msgr2=0x7fb7900cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:47.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 shutdown_connections 2024-09-15T23:37:47.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.522+0000 7fb794d66640 1 -- 172.21.15.7:0/1048741401 wait complete. 2024-09-15T23:37:47.535 INFO:tasks.ceph:Scrubbing osd.6 2024-09-15T23:37:47.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-15T23:37:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.640+0000 7f39af926640 1 Processor -- start 2024-09-15T23:37:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.640+0000 7f39af926640 1 -- start start 2024-09-15T23:37:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.641+0000 7f39af926640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f39a815ebd0 0x7f39a80b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.641+0000 7f39af926640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a809b5a0 con 0x7f39a809b150 2024-09-15T23:37:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.641+0000 7f39af926640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a80b3d10 con 0x7f39a80b38d0 2024-09-15T23:37:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.641+0000 7f39af926640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f39a80b3640 con 0x7f39a815ebd0 2024-09-15T23:37:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.641+0000 7f39ad6c2640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f39a815ebd0 0x7f39a80b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.641+0000 7f39adec3640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f39a809b150 0x7f39a806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38644/0 (socket says 172.21.15.7:38644) 2024-09-15T23:37:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.641+0000 7f39adec3640 1 -- 172.21.15.7:0/3579961391 learned_addr learned my addr 172.21.15.7:0/3579961391 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:37:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.642+0000 7f39ad6c2640 1 -- 172.21.15.7:0/3579961391 >> v1:172.21.15.18:6789/0 conn(0x7f39a80b38d0 legacy=0x7f39a809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.642+0000 7f39ad6c2640 1 -- 172.21.15.7:0/3579961391 >> v1:172.21.15.7:6789/0 conn(0x7f39a809b150 legacy=0x7f39a806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.642+0000 7f39ad6c2640 1 -- 172.21.15.7:0/3579961391 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80b2d00 con 0x7f39a815ebd0 2024-09-15T23:37:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.642+0000 7f39ad6c2640 1 --2- 172.21.15.7:0/3579961391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 0x7f39a80b4a60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f39a40097a0 tx=0x7f39a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5698b0caaa9c5b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f399bfff640 1 -- 172.21.15.7:0/3579961391 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4037070 con 0x7f39a815ebd0 2024-09-15T23:37:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f399bfff640 1 -- 172.21.15.7:0/3579961391 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a4032440 con 0x7f39a815ebd0 2024-09-15T23:37:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f399bfff640 1 -- 172.21.15.7:0/3579961391 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4032810 con 0x7f39a815ebd0 2024-09-15T23:37:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f39af926640 1 -- 172.21.15.7:0/3579961391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 msgr2=0x7f39a80b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f39af926640 1 --2- 172.21.15.7:0/3579961391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 0x7f39a80b4a60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f39a40097a0 tx=0x7f39a402a1e0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f39af926640 1 -- 172.21.15.7:0/3579961391 shutdown_connections 2024-09-15T23:37:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f39af926640 1 --2- 172.21.15.7:0/3579961391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 0x7f39a80b4a60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f39af926640 1 -- 172.21.15.7:0/3579961391 >> 172.21.15.7:0/3579961391 conn(0x7f39a80b3f40 msgr2=0x7f39a80b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f39af926640 1 -- 172.21.15.7:0/3579961391 shutdown_connections 2024-09-15T23:37:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.643+0000 7f39af926640 1 -- 172.21.15.7:0/3579961391 wait complete. 2024-09-15T23:37:47.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.644+0000 7f39af926640 1 Processor -- start 2024-09-15T23:37:47.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39af926640 1 -- start start 2024-09-15T23:37:47.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39af926640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39a809b150 0x7f39a8150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39af926640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39a80b38d0 0x7f39a80c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39af926640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 0x7f39a815d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39af926640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39a809b850 con 0x7f39a815ebd0 2024-09-15T23:37:47.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39af926640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39a809bbc0 con 0x7f39a809b150 2024-09-15T23:37:47.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39af926640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f39a815efb0 con 0x7f39a80b38d0 2024-09-15T23:37:47.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39adec3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 0x7f39a815d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39adec3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 0x7f39a815d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51698/0 (socket says 172.21.15.7:51698) 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.645+0000 7f39adec3640 1 -- 172.21.15.7:0/6650920 learned_addr learned my addr 172.21.15.7:0/6650920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39ad6c2640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39a809b150 0x7f39a8150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39adec3640 1 -- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39a80b38d0 msgr2=0x7f39a80c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39acec1640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39a80b38d0 0x7f39a80c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39adec3640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39a80b38d0 0x7f39a80c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39adec3640 1 -- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39a809b150 msgr2=0x7f39a8150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39adec3640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39a809b150 0x7f39a8150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39adec3640 1 -- 172.21.15.7:0/6650920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80b0fb0 con 0x7f39a815ebd0 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39acec1640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39a80b38d0 0x7f39a80c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39ad6c2640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39a809b150 0x7f39a8150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39adec3640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 0x7f39a815d470 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f39a000ca80 tx=0x7f39a000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f3999ffb640 1 -- 172.21.15.7:0/6650920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0017070 con 0x7f39a815ebd0 2024-09-15T23:37:47.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f3999ffb640 1 -- 172.21.15.7:0/6650920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a000ac30 con 0x7f39a815ebd0 2024-09-15T23:37:47.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a8150570 con 0x7f39a815ebd0 2024-09-15T23:37:47.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f3999ffb640 1 -- 172.21.15.7:0/6650920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0012400 con 0x7f39a815ebd0 2024-09-15T23:37:47.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.646+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a8153080 con 0x7f39a815ebd0 2024-09-15T23:37:47.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.647+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39a806bfe0 con 0x7f39a815ebd0 2024-09-15T23:37:47.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.648+0000 7f3999ffb640 1 -- 172.21.15.7:0/6650920 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f39a001a070 con 0x7f39a815ebd0 2024-09-15T23:37:47.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.648+0000 7f3999ffb640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f398c06c2b0 0x7f398c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.648+0000 7f3999ffb640 1 -- 172.21.15.7:0/6650920 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f39a008b0f0 con 0x7f39a815ebd0 2024-09-15T23:37:47.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.648+0000 7f39ad6c2640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f398c06c2b0 0x7f398c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.648+0000 7f3999ffb640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7f398c072ca0 0x7f398c0750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.648+0000 7f3999ffb640 1 -- 172.21.15.7:0/6650920 --> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f398c0757b0 con 0x7f398c072ca0 2024-09-15T23:37:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.648+0000 7f3999ffb640 1 -- 172.21.15.7:0/6650920 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f39a0092050 con 0x7f39a815ebd0 2024-09-15T23:37:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.649+0000 7f39ad6c2640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f398c06c2b0 0x7f398c06e770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39a4034000 tx=0x7f39a402e000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.649+0000 7f39acec1640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7f398c072ca0 0x7f398c0750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.649+0000 7f39acec1640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7f398c072ca0 0x7f398c0750e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.650+0000 7f3999ffb640 1 -- 172.21.15.7:0/6650920 <== osd.6 v2:172.21.15.18:6824/468690220 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f398c0757b0 con 0x7f398c072ca0 2024-09-15T23:37:47.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.663+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 --> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f39a8076060 con 0x7f398c072ca0 2024-09-15T23:37:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.663+0000 7f3999ffb640 1 -- 172.21.15.7:0/6650920 <== osd.6 v2:172.21.15.18:6824/468690220 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f39a8076060 con 0x7f398c072ca0 2024-09-15T23:37:47.664 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T23:37:47.664 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.663+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7f398c072ca0 msgr2=0x7f398c0750e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.663+0000 7f39af926640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.18:6824/468690220,v1:172.21.15.18:6825/468690220] conn(0x7f398c072ca0 0x7f398c0750e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.664+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f398c06c2b0 msgr2=0x7f398c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.664+0000 7f39af926640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f398c06c2b0 0x7f398c06e770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39a4034000 tx=0x7f39a402e000 comp rx=0 tx=0).stop 2024-09-15T23:37:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.664+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 msgr2=0x7f39a815d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.664+0000 7f39af926640 1 --2- 172.21.15.7:0/6650920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815ebd0 0x7f39a815d470 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f39a000ca80 tx=0x7f39a000cf40 comp rx=0 tx=0).stop 2024-09-15T23:37:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.664+0000 7f39adec3640 1 -- 172.21.15.7:0/6650920 reap_dead start 2024-09-15T23:37:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.664+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 shutdown_connections 2024-09-15T23:37:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.664+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 >> 172.21.15.7:0/6650920 conn(0x7f39a80b3f40 msgr2=0x7f39a8070110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.664+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 shutdown_connections 2024-09-15T23:37:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.664+0000 7f39af926640 1 -- 172.21.15.7:0/6650920 wait complete. 2024-09-15T23:37:47.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-15T23:37:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.777+0000 7f7ebb59e640 1 Processor -- start 2024-09-15T23:37:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.777+0000 7f7ebb59e640 1 -- start start 2024-09-15T23:37:47.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.778+0000 7f7ebb59e640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ebc15b100 0x7f7ebc06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.778+0000 7f7ebb59e640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ebc0aeba0 con 0x7f7ebc0ae710 2024-09-15T23:37:47.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.778+0000 7f7ebb59e640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ebc0aee80 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.778+0000 7f7ebb59e640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7ebc15c020 con 0x7f7ebc15b100 2024-09-15T23:37:47.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.778+0000 7f7eba59c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ebc15b100 0x7f7ebc06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.778+0000 7f7eba59c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ebc15b100 0x7f7ebc06dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51708/0 (socket says 172.21.15.7:51708) 2024-09-15T23:37:47.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.778+0000 7f7eb9d9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7ebc0ae710 0x7f7ebc15b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38660/0 (socket says 172.21.15.7:38660) 2024-09-15T23:37:47.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.778+0000 7f7eba59c640 1 -- 172.21.15.7:0/2739773273 learned_addr learned my addr 172.21.15.7:0/2739773273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:47.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.779+0000 7f7eba59c640 1 -- 172.21.15.7:0/2739773273 >> v1:172.21.15.18:6789/0 conn(0x7f7ebc15bbf0 legacy=0x7f7ebc0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:47.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.779+0000 7f7eba59c640 1 -- 172.21.15.7:0/2739773273 >> v1:172.21.15.7:6789/0 conn(0x7f7ebc0ae710 legacy=0x7f7ebc15b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.779+0000 7f7eba59c640 1 -- 172.21.15.7:0/2739773273 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc0af160 con 0x7f7ebc15b100 2024-09-15T23:37:47.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.779+0000 7f7eba59c640 1 --2- 172.21.15.7:0/2739773273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15b100 0x7f7ebc06dd50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009870 tx=0x7f7eac02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f6f12aeace7781 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7eb959a640 1 -- 172.21.15.7:0/2739773273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac037070 con 0x7f7ebc15b100 2024-09-15T23:37:47.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7eb959a640 1 -- 172.21.15.7:0/2739773273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eac032440 con 0x7f7ebc15b100 2024-09-15T23:37:47.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7eb959a640 1 -- 172.21.15.7:0/2739773273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac032810 con 0x7f7ebc15b100 2024-09-15T23:37:47.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7ebb59e640 1 -- 172.21.15.7:0/2739773273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15b100 msgr2=0x7f7ebc06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7ebb59e640 1 --2- 172.21.15.7:0/2739773273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15b100 0x7f7ebc06dd50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009870 tx=0x7f7eac02a2a0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7ebb59e640 1 -- 172.21.15.7:0/2739773273 shutdown_connections 2024-09-15T23:37:47.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7ebb59e640 1 --2- 172.21.15.7:0/2739773273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15b100 0x7f7ebc06dd50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7ebb59e640 1 -- 172.21.15.7:0/2739773273 >> 172.21.15.7:0/2739773273 conn(0x7f7ebc0b1be0 msgr2=0x7f7ebc0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:47.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7ebb59e640 1 -- 172.21.15.7:0/2739773273 shutdown_connections 2024-09-15T23:37:47.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.780+0000 7f7ebb59e640 1 -- 172.21.15.7:0/2739773273 wait complete. 2024-09-15T23:37:47.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.782+0000 7f7ebb59e640 1 Processor -- start 2024-09-15T23:37:47.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.782+0000 7f7ebb59e640 1 -- start start 2024-09-15T23:37:47.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.782+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ebc15b100 0x7f7ebc15df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.782+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15bbf0 0x7f7ebc158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.782+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7ebc09d1d0 0x7f7ebc15d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.782+0000 7f7ebb59e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc06e290 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.782+0000 7f7ebb59e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc0b3260 con 0x7f7ebc15b100 2024-09-15T23:37:47.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.782+0000 7f7ebb59e640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7ebc06e410 con 0x7f7ebc09d1d0 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.782+0000 7f7eb9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15bbf0 0x7f7ebc158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eb9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15bbf0 0x7f7ebc158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51722/0 (socket says 172.21.15.7:51722) 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eb9d9b640 1 -- 172.21.15.7:0/3048760561 learned_addr learned my addr 172.21.15.7:0/3048760561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eb9d9b640 1 -- 172.21.15.7:0/3048760561 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7ebc09d1d0 msgr2=0x7f7ebc15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eb9d9b640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7ebc09d1d0 0x7f7ebc15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eb9d9b640 1 -- 172.21.15.7:0/3048760561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ebc15b100 msgr2=0x7f7ebc15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eb9d9b640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ebc15b100 0x7f7ebc15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eb9d9b640 1 -- 172.21.15.7:0/3048760561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc15e560 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eb9d9b640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15bbf0 0x7f7ebc158f30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000c400 tx=0x7f7eb000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eaaffd640 1 -- 172.21.15.7:0/3048760561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00074e0 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eaaffd640 1 -- 172.21.15.7:0/3048760561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb0007680 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc09c6f0 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7eaaffd640 1 -- 172.21.15.7:0/3048760561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0004a00 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.783+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc14dba0 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.784+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc06dd50 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.785+0000 7f7eaaffd640 1 -- 172.21.15.7:0/3048760561 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f7eb001a020 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.785+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7e9c06c2b0 0x7f7e9c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.785+0000 7f7eaaffd640 1 -- 172.21.15.7:0/3048760561 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f7eb008ad80 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.785+0000 7f7eba59c640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7e9c06c2b0 0x7f7e9c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.786+0000 7f7eba59c640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7e9c06c2b0 0x7f7e9c06e770 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7eac034000 tx=0x7f7eac02c000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.790+0000 7f7eaaffd640 1 -- 172.21.15.7:0/3048760561 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7eb0059400 con 0x7f7ebc15bbf0 2024-09-15T23:37:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.967+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f7ebc0afa50 con 0x7f7e9c06c2b0 2024-09-15T23:37:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.967+0000 7f7eaaffd640 1 -- 172.21.15.7:0/3048760561 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7ebc0afa50 con 0x7f7e9c06c2b0 2024-09-15T23:37:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2024-09-15T23:37:47.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7e9c06c2b0 msgr2=0x7f7e9c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7e9c06c2b0 0x7f7e9c06e770 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7eac034000 tx=0x7f7eac02c000 comp rx=0 tx=0).stop 2024-09-15T23:37:47.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15bbf0 msgr2=0x7f7ebc158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:47.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15bbf0 0x7f7ebc158f30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000c400 tx=0x7f7eb000c8c0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 shutdown_connections 2024-09-15T23:37:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7e9c06c2b0 0x7f7e9c06e770 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7ebc09d1d0 0x7f7ebc15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ebc15bbf0 0x7f7ebc158f30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 --2- 172.21.15.7:0/3048760561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ebc15b100 0x7f7ebc15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 >> 172.21.15.7:0/3048760561 conn(0x7f7ebc0b1be0 msgr2=0x7f7ebc0adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.969+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 shutdown_connections 2024-09-15T23:37:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:47.970+0000 7f7ebb59e640 1 -- 172.21.15.7:0/3048760561 wait complete. 2024-09-15T23:37:47.981 INFO:tasks.ceph:Scrubbing osd.7 2024-09-15T23:37:47.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-15T23:37:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.086+0000 7f1c17a8d640 1 Processor -- start 2024-09-15T23:37:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.087+0000 7f1c17a8d640 1 -- start start 2024-09-15T23:37:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.087+0000 7f1c17a8d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c1015e5b0 0x7f1c10162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.087+0000 7f1c17a8d640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c100d24e0 0x7f1c10162890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.087+0000 7f1c17a8d640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c100d2980 con 0x7f1c1015e990 2024-09-15T23:37:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.087+0000 7f1c17a8d640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1c100d2c40 con 0x7f1c100d24e0 2024-09-15T23:37:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.087+0000 7f1c17a8d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f1c100d2f20 con 0x7f1c1015e5b0 2024-09-15T23:37:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.088+0000 7f1c15028640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c100d24e0 0x7f1c10162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.088+0000 7f1c15028640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c100d24e0 0x7f1c10162890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51730/0 (socket says 172.21.15.7:51730) 2024-09-15T23:37:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.088+0000 7f1c15028640 1 -- 172.21.15.7:0/207951866 learned_addr learned my addr 172.21.15.7:0/207951866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.088+0000 7f1c15829640 1 --2- 172.21.15.7:0/207951866 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c1015e5b0 0x7f1c10162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.088+0000 7f1c15028640 1 -- 172.21.15.7:0/207951866 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c1015e5b0 msgr2=0x7f1c10162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.088+0000 7f1c15028640 1 --2- 172.21.15.7:0/207951866 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c1015e5b0 0x7f1c10162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.088+0000 7f1c15028640 1 -- 172.21.15.7:0/207951866 >> v1:172.21.15.18:6789/0 conn(0x7f1c1015e990 legacy=0x7f1c100d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.088+0000 7f1c15028640 1 -- 172.21.15.7:0/207951866 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c100d3200 con 0x7f1c100d24e0 2024-09-15T23:37:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.089+0000 7f1c15028640 1 --2- 172.21.15.7:0/207951866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c100d24e0 0x7f1c10162890 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009850 tx=0x7f1c0402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4241af3ec48a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.089+0000 7f1c14827640 1 -- 172.21.15.7:0/207951866 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04004030 con 0x7f1c100d24e0 2024-09-15T23:37:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.089+0000 7f1c14827640 1 -- 172.21.15.7:0/207951866 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c040041d0 con 0x7f1c100d24e0 2024-09-15T23:37:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.089+0000 7f1c14827640 1 -- 172.21.15.7:0/207951866 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04003610 con 0x7f1c100d24e0 2024-09-15T23:37:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.089+0000 7f1c17a8d640 1 -- 172.21.15.7:0/207951866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c100d24e0 msgr2=0x7f1c10162890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.089+0000 7f1c17a8d640 1 --2- 172.21.15.7:0/207951866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c100d24e0 0x7f1c10162890 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009850 tx=0x7f1c0402a230 comp rx=0 tx=0).stop 2024-09-15T23:37:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.090+0000 7f1c17a8d640 1 -- 172.21.15.7:0/207951866 shutdown_connections 2024-09-15T23:37:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.090+0000 7f1c17a8d640 1 --2- 172.21.15.7:0/207951866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c100d24e0 0x7f1c10162890 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.090+0000 7f1c17a8d640 1 --2- 172.21.15.7:0/207951866 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c1015e5b0 0x7f1c10162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.090+0000 7f1c17a8d640 1 -- 172.21.15.7:0/207951866 >> 172.21.15.7:0/207951866 conn(0x7f1c1008bfd0 msgr2=0x7f1c100d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.090+0000 7f1c17a8d640 1 -- 172.21.15.7:0/207951866 shutdown_connections 2024-09-15T23:37:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.090+0000 7f1c17a8d640 1 -- 172.21.15.7:0/207951866 wait complete. 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.091+0000 7f1c17a8d640 1 Processor -- start 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.091+0000 7f1c17a8d640 1 -- start start 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.091+0000 7f1c17a8d640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c100d24e0 0x7f1c100b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c17a8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c1015e5b0 0x7f1c10140710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c17a8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015e990 0x7f1c1006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c17a8d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c100b3b40 con 0x7f1c1015e990 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c17a8d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1c100b39c0 con 0x7f1c1015e5b0 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c17a8d640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f1c100d0430 con 0x7f1c100d24e0 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c1602a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015e990 0x7f1c1006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c1602a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015e990 0x7f1c1006d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51732/0 (socket says 172.21.15.7:51732) 2024-09-15T23:37:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c1602a640 1 -- 172.21.15.7:0/378722586 learned_addr learned my addr 172.21.15.7:0/378722586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c15028640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c1015e5b0 0x7f1c10140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c15829640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c100d24e0 0x7f1c100b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c1602a640 1 -- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c100d24e0 msgr2=0x7f1c100b8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c1602a640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1c100d24e0 0x7f1c100b8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c1602a640 1 -- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c1015e5b0 msgr2=0x7f1c10140710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c1602a640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c1015e5b0 0x7f1c10140710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c1602a640 1 -- 172.21.15.7:0/378722586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c100b9420 con 0x7f1c1015e990 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c1602a640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015e990 0x7f1c1006d540 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800c3c0 tx=0x7f1c0800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c0e7fc640 1 -- 172.21.15.7:0/378722586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08017070 con 0x7f1c1015e990 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c0e7fc640 1 -- 172.21.15.7:0/378722586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c080052d0 con 0x7f1c1015e990 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c100b9700 con 0x7f1c1015e990 2024-09-15T23:37:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.092+0000 7f1c0e7fc640 1 -- 172.21.15.7:0/378722586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08003830 con 0x7f1c1015e990 2024-09-15T23:37:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.093+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c100b9cc0 con 0x7f1c1015e990 2024-09-15T23:37:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.093+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c10162350 con 0x7f1c1015e990 2024-09-15T23:37:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.094+0000 7f1c0e7fc640 1 -- 172.21.15.7:0/378722586 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f1c0801a070 con 0x7f1c1015e990 2024-09-15T23:37:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.094+0000 7f1c0e7fc640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1bfc06c2b0 0x7f1bfc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.094+0000 7f1c0e7fc640 1 -- 172.21.15.7:0/378722586 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f1c0808a8c0 con 0x7f1c1015e990 2024-09-15T23:37:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.094+0000 7f1c15829640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1bfc06c2b0 0x7f1bfc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.094+0000 7f1c0e7fc640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f1bfc072ca0 0x7f1bfc0750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.095+0000 7f1c0e7fc640 1 -- 172.21.15.7:0/378722586 --> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c08015e50 con 0x7f1bfc072ca0 2024-09-15T23:37:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.095+0000 7f1c0e7fc640 1 -- 172.21.15.7:0/378722586 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1c0808f2c0 con 0x7f1c1015e990 2024-09-15T23:37:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.095+0000 7f1c15028640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f1bfc072ca0 0x7f1bfc0750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.095+0000 7f1c15829640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1bfc06c2b0 0x7f1bfc06e770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c00003b80 tx=0x7f1c000098a0 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.095+0000 7f1c15028640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f1bfc072ca0 0x7f1bfc0750e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.096+0000 7f1c0e7fc640 1 -- 172.21.15.7:0/378722586 <== osd.7 v2:172.21.15.18:6816/1769482769 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c08015e50 con 0x7f1bfc072ca0 2024-09-15T23:37:48.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.109+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 --> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1c100b3570 con 0x7f1bfc072ca0 2024-09-15T23:37:48.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.109+0000 7f1c0e7fc640 1 -- 172.21.15.7:0/378722586 <== osd.7 v2:172.21.15.18:6816/1769482769 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1c100b3570 con 0x7f1bfc072ca0 2024-09-15T23:37:48.110 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-15T23:37:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T23:37:48.110 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-15T23:37:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.109+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f1bfc072ca0 msgr2=0x7f1bfc0750e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.109+0000 7f1c17a8d640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.18:6816/1769482769,v1:172.21.15.18:6817/1769482769] conn(0x7f1bfc072ca0 0x7f1bfc0750e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.109+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1bfc06c2b0 msgr2=0x7f1bfc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.109+0000 7f1c17a8d640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f1bfc06c2b0 0x7f1bfc06e770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c00003b80 tx=0x7f1c000098a0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.109+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015e990 msgr2=0x7f1c1006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.109+0000 7f1c17a8d640 1 --2- 172.21.15.7:0/378722586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015e990 0x7f1c1006d540 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800c3c0 tx=0x7f1c0800c7c0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.109+0000 7f1c1602a640 1 -- 172.21.15.7:0/378722586 reap_dead start 2024-09-15T23:37:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.110+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 shutdown_connections 2024-09-15T23:37:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.110+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 >> 172.21.15.7:0/378722586 conn(0x7f1c1008bfd0 msgr2=0x7f1c1014e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.110+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 shutdown_connections 2024-09-15T23:37:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.110+0000 7f1c17a8d640 1 -- 172.21.15.7:0/378722586 wait complete. 2024-09-15T23:37:48.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-15T23:37:48.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.223+0000 7fe2316e6640 1 Processor -- start 2024-09-15T23:37:48.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.223+0000 7fe2316e6640 1 -- start start 2024-09-15T23:37:48.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.224+0000 7fe2316e6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 0x7fe22c06c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.224+0000 7fe2316e6640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe22c15f3e0 0x7fe22c0aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.224+0000 7fe2316e6640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe22c0b44d0 con 0x7fe22c162920 2024-09-15T23:37:48.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.224+0000 7fe2316e6640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe22c06cb60 con 0x7fe22c15f3e0 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.224+0000 7fe2316e6640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c0b6560 con 0x7fe22c162540 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.224+0000 7fe22b7fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe22c15f3e0 0x7fe22c0aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.224+0000 7fe22b7fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe22c15f3e0 0x7fe22c0aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51744/0 (socket says 172.21.15.7:51744) 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.224+0000 7fe22b7fe640 1 -- 172.21.15.7:0/2228779962 learned_addr learned my addr 172.21.15.7:0/2228779962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.225+0000 7fe22affd640 1 --2- 172.21.15.7:0/2228779962 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 0x7fe22c06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.225+0000 7fe22b7fe640 1 -- 172.21.15.7:0/2228779962 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 msgr2=0x7fe22c06c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.225+0000 7fe22b7fe640 1 --2- 172.21.15.7:0/2228779962 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 0x7fe22c06c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.225+0000 7fe22b7fe640 1 -- 172.21.15.7:0/2228779962 >> v1:172.21.15.18:6789/0 conn(0x7fe22c162920 legacy=0x7fe22c0ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.225+0000 7fe22b7fe640 1 -- 172.21.15.7:0/2228779962 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c0b4740 con 0x7fe22c15f3e0 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.225+0000 7fe22affd640 1 --2- 172.21.15.7:0/2228779962 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 0x7fe22c06c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.225+0000 7fe22b7fe640 1 --2- 172.21.15.7:0/2228779962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 0x7fe22c0aebb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe224009870 tx=0x7fe22402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c215282ef03e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe229ffb640 1 -- 172.21.15.7:0/2228779962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224037070 con 0x7fe22c15f3e0 2024-09-15T23:37:48.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe229ffb640 1 -- 172.21.15.7:0/2228779962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe224032440 con 0x7fe22c15f3e0 2024-09-15T23:37:48.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe229ffb640 1 -- 172.21.15.7:0/2228779962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224032810 con 0x7fe22c15f3e0 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe2316e6640 1 -- 172.21.15.7:0/2228779962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 msgr2=0x7fe22c0aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe2316e6640 1 --2- 172.21.15.7:0/2228779962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 0x7fe22c0aebb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe224009870 tx=0x7fe22402a180 comp rx=0 tx=0).stop 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe2316e6640 1 -- 172.21.15.7:0/2228779962 shutdown_connections 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe2316e6640 1 --2- 172.21.15.7:0/2228779962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 0x7fe22c0aebb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe2316e6640 1 --2- 172.21.15.7:0/2228779962 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 0x7fe22c06c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe2316e6640 1 -- 172.21.15.7:0/2228779962 >> 172.21.15.7:0/2228779962 conn(0x7fe22c0b70c0 msgr2=0x7fe22c09aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe2316e6640 1 -- 172.21.15.7:0/2228779962 shutdown_connections 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.226+0000 7fe2316e6640 1 -- 172.21.15.7:0/2228779962 wait complete. 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.227+0000 7fe2316e6640 1 Processor -- start 2024-09-15T23:37:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe2316e6640 1 -- start start 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe2316e6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 0x7fe22c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe2316e6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 0x7fe22c156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe2316e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe22c162920 0x7fe22c154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe2316e6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c163140 con 0x7fe22c15f3e0 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe2316e6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c162fc0 con 0x7fe22c162920 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe2316e6640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c0b6130 con 0x7fe22c162540 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe22affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 0x7fe22c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe22affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 0x7fe22c153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51746/0 (socket says 172.21.15.7:51746) 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.228+0000 7fe22affd640 1 -- 172.21.15.7:0/4245973258 learned_addr learned my addr 172.21.15.7:0/4245973258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe22b7fe640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe22c162920 0x7fe22c154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe22a7fc640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 0x7fe22c156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe22affd640 1 -- 172.21.15.7:0/4245973258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 msgr2=0x7fe22c156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe22affd640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 0x7fe22c156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe22affd640 1 -- 172.21.15.7:0/4245973258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe22c162920 msgr2=0x7fe22c154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe22affd640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe22c162920 0x7fe22c154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe22affd640 1 -- 172.21.15.7:0/4245973258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c156d20 con 0x7fe22c15f3e0 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe22b7fe640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe22c162920 0x7fe22c154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe22affd640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 0x7fe22c153a10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00cf60 tx=0x7fe21c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe2237fe640 1 -- 172.21.15.7:0/4245973258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c01a070 con 0x7fe22c15f3e0 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe2237fe640 1 -- 172.21.15.7:0/4245973258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21c00d840 con 0x7fe22c15f3e0 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c1f4870 con 0x7fe22c15f3e0 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe2237fe640 1 -- 172.21.15.7:0/4245973258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00dc10 con 0x7fe22c15f3e0 2024-09-15T23:37:48.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.229+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c1f4e30 con 0x7fe22c15f3e0 2024-09-15T23:37:48.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.230+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c15f850 con 0x7fe22c15f3e0 2024-09-15T23:37:48.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.231+0000 7fe2237fe640 1 -- 172.21.15.7:0/4245973258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fe21c01d070 con 0x7fe22c15f3e0 2024-09-15T23:37:48.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.231+0000 7fe2237fe640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe20c06c2b0 0x7fe20c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.231+0000 7fe2237fe640 1 -- 172.21.15.7:0/4245973258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fe21c08bd90 con 0x7fe22c15f3e0 2024-09-15T23:37:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.231+0000 7fe22a7fc640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe20c06c2b0 0x7fe20c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.232+0000 7fe22a7fc640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe20c06c2b0 0x7fe20c06e770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe214006fd0 tx=0x7fe214008040 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.235+0000 7fe2237fe640 1 -- 172.21.15.7:0/4245973258 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe21c059d40 con 0x7fe22c15f3e0 2024-09-15T23:37:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.414+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fe22c06c840 con 0x7fe20c06c2b0 2024-09-15T23:37:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.414+0000 7fe2237fe640 1 -- 172.21.15.7:0/4245973258 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe22c06c840 con 0x7fe20c06c2b0 2024-09-15T23:37:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2024-09-15T23:37:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe20c06c2b0 msgr2=0x7fe20c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe20c06c2b0 0x7fe20c06e770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe214006fd0 tx=0x7fe214008040 comp rx=0 tx=0).stop 2024-09-15T23:37:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 msgr2=0x7fe22c153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 0x7fe22c153a10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00cf60 tx=0x7fe21c00cf90 comp rx=0 tx=0).stop 2024-09-15T23:37:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 shutdown_connections 2024-09-15T23:37:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fe20c06c2b0 0x7fe20c06e770 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe22c162920 0x7fe22c154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe22c162540 0x7fe22c156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 --2- 172.21.15.7:0/4245973258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22c15f3e0 0x7fe22c153a10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 >> 172.21.15.7:0/4245973258 conn(0x7fe22c0b70c0 msgr2=0x7fe22c159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 shutdown_connections 2024-09-15T23:37:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.416+0000 7fe2316e6640 1 -- 172.21.15.7:0/4245973258 wait complete. 2024-09-15T23:37:48.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:37:48.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.529+0000 7fbb43f66640 1 Processor -- start 2024-09-15T23:37:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.529+0000 7fbb43f66640 1 -- start start 2024-09-15T23:37:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb43f66640 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbb3c0b31c0 0x7fbb3c0d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb43f66640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb3c15ca20 0x7fbb3c0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb43f66640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3c06dfa0 con 0x7fbb3c158e60 2024-09-15T23:37:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb43f66640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbb3c159240 con 0x7fbb3c15ca20 2024-09-15T23:37:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb43f66640 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 -- 0x7fbb3c06e210 con 0x7fbb3c0b31c0 2024-09-15T23:37:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb42503640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb3c15ca20 0x7fbb3c0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb42503640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb3c15ca20 0x7fbb3c0d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51748/0 (socket says 172.21.15.7:51748) 2024-09-15T23:37:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb42503640 1 -- 172.21.15.7:0/298144937 learned_addr learned my addr 172.21.15.7:0/298144937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb41501640 1 --2- 172.21.15.7:0/298144937 >> v2:172.21.15.18:3300/0 conn(0x7fbb3c0b31c0 0x7fbb3c0d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb42503640 1 -- 172.21.15.7:0/298144937 >> v2:172.21.15.18:3300/0 conn(0x7fbb3c0b31c0 msgr2=0x7fbb3c0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb42503640 1 --2- 172.21.15.7:0/298144937 >> v2:172.21.15.18:3300/0 conn(0x7fbb3c0b31c0 0x7fbb3c0d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb42503640 1 -- 172.21.15.7:0/298144937 >> v1:172.21.15.18:6789/0 conn(0x7fbb3c158e60 legacy=0x7fbb3c0b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:37:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb42503640 1 -- 172.21.15.7:0/298144937 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c06e4f0 con 0x7fbb3c15ca20 2024-09-15T23:37:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.530+0000 7fbb41501640 1 --2- 172.21.15.7:0/298144937 >> v2:172.21.15.18:3300/0 conn(0x7fbb3c0b31c0 0x7fbb3c0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:37:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.531+0000 7fbb42503640 1 --2- 172.21.15.7:0/298144937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c15ca20 0x7fbb3c0d0b30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c15b860 tx=0x7fbb3402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c9bb2993180519 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.531+0000 7fbb40d00640 1 -- 172.21.15.7:0/298144937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34036070 con 0x7fbb3c15ca20 2024-09-15T23:37:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.531+0000 7fbb40d00640 1 -- 172.21.15.7:0/298144937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb34031600 con 0x7fbb3c15ca20 2024-09-15T23:37:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.531+0000 7fbb40d00640 1 -- 172.21.15.7:0/298144937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb340319d0 con 0x7fbb3c15ca20 2024-09-15T23:37:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.532+0000 7fbb43f66640 1 -- 172.21.15.7:0/298144937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c15ca20 msgr2=0x7fbb3c0d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.532+0000 7fbb43f66640 1 --2- 172.21.15.7:0/298144937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c15ca20 0x7fbb3c0d0b30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c15b860 tx=0x7fbb3402a780 comp rx=0 tx=0).stop 2024-09-15T23:37:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.532+0000 7fbb43f66640 1 -- 172.21.15.7:0/298144937 shutdown_connections 2024-09-15T23:37:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.532+0000 7fbb43f66640 1 --2- 172.21.15.7:0/298144937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c15ca20 0x7fbb3c0d0b30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.532+0000 7fbb43f66640 1 --2- 172.21.15.7:0/298144937 >> v2:172.21.15.18:3300/0 conn(0x7fbb3c0b31c0 0x7fbb3c0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.532+0000 7fbb43f66640 1 -- 172.21.15.7:0/298144937 >> 172.21.15.7:0/298144937 conn(0x7fbb3c0b21e0 msgr2=0x7fbb3c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.532+0000 7fbb43f66640 1 -- 172.21.15.7:0/298144937 shutdown_connections 2024-09-15T23:37:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.532+0000 7fbb43f66640 1 -- 172.21.15.7:0/298144937 wait complete. 2024-09-15T23:37:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.533+0000 7fbb43f66640 1 Processor -- start 2024-09-15T23:37:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.533+0000 7fbb43f66640 1 -- start start 2024-09-15T23:37:48.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb43f66640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb3c0b31c0 0x7fbb3c148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb43f66640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c158e60 0x7fbb3c09a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb43f66640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbb3c15ca20 0x7fbb3c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb43f66640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c0b3d90 con 0x7fbb3c158e60 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb43f66640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c06eab0 con 0x7fbb3c0b31c0 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb43f66640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c0b3f10 con 0x7fbb3c15ca20 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41501640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c158e60 0x7fbb3c09a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41501640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c158e60 0x7fbb3c09a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51756/0 (socket says 172.21.15.7:51756) 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41501640 1 -- 172.21.15.7:0/3541352910 learned_addr learned my addr 172.21.15.7:0/3541352910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41d02640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb3c0b31c0 0x7fbb3c148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb42503640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbb3c15ca20 0x7fbb3c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41501640 1 -- 172.21.15.7:0/3541352910 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbb3c15ca20 msgr2=0x7fbb3c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41501640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbb3c15ca20 0x7fbb3c0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41501640 1 -- 172.21.15.7:0/3541352910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb3c0b31c0 msgr2=0x7fbb3c148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41501640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb3c0b31c0 0x7fbb3c148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41501640 1 -- 172.21.15.7:0/3541352910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c09d4c0 con 0x7fbb3c158e60 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb41501640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c158e60 0x7fbb3c09a1b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c7f0 tx=0x7fbb3000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.534+0000 7fbb2e7fc640 1 -- 172.21.15.7:0/3541352910 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30015070 con 0x7fbb3c158e60 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.535+0000 7fbb2e7fc640 1 -- 172.21.15.7:0/3541352910 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb300077c0 con 0x7fbb3c158e60 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.535+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c09d7a0 con 0x7fbb3c158e60 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.535+0000 7fbb2e7fc640 1 -- 172.21.15.7:0/3541352910 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30007b90 con 0x7fbb3c158e60 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.535+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c09dd60 con 0x7fbb3c158e60 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.536+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c0b3680 con 0x7fbb3c158e60 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.536+0000 7fbb2e7fc640 1 -- 172.21.15.7:0/3541352910 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fbb30018070 con 0x7fbb3c158e60 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.536+0000 7fbb2e7fc640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbb2006c2e0 0x7fbb2006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.537+0000 7fbb2e7fc640 1 -- 172.21.15.7:0/3541352910 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fbb3008b520 con 0x7fbb3c158e60 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.537+0000 7fbb41d02640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbb2006c2e0 0x7fbb2006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:37:48.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.537+0000 7fbb41d02640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbb2006c2e0 0x7fbb2006e7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb38004f60 tx=0x7fbb38007e30 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:37:48.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.541+0000 7fbb2e7fc640 1 -- 172.21.15.7:0/3541352910 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb30059550 con 0x7fbb3c158e60 2024-09-15T23:37:48.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.714+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb3c15a610 con 0x7fbb2006c2e0 2024-09-15T23:37:48.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.714+0000 7fbb2e7fc640 1 -- 172.21.15.7:0/3541352910 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13465 (secure 0 0 0) 0x7fbb3c15a610 con 0x7fbb2006c2e0 2024-09-15T23:37:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:37:48.716 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:37:48.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.716+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbb2006c2e0 msgr2=0x7fbb2006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.716+0000 7fbb43f66640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbb2006c2e0 0x7fbb2006e7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb38004f60 tx=0x7fbb38007e30 comp rx=0 tx=0).stop 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.716+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c158e60 msgr2=0x7fbb3c09a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.716+0000 7fbb43f66640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c158e60 0x7fbb3c09a1b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c7f0 tx=0x7fbb3000cbf0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.717+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 shutdown_connections 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.717+0000 7fbb43f66640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fbb2006c2e0 0x7fbb2006e7a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.717+0000 7fbb43f66640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbb3c15ca20 0x7fbb3c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.717+0000 7fbb43f66640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3c158e60 0x7fbb3c09a1b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.717+0000 7fbb43f66640 1 --2- 172.21.15.7:0/3541352910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb3c0b31c0 0x7fbb3c148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.717+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 >> 172.21.15.7:0/3541352910 conn(0x7fbb3c0b21e0 msgr2=0x7fbb3c159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.717+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 shutdown_connections 2024-09-15T23:37:48.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:37:48.717+0000 7fbb43f66640 1 -- 172.21.15.7:0/3541352910 wait complete. 2024-09-15T23:37:48.728 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-15T23:37:47.330068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":254696,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720024,"statfs":{"total":773094113280,"available":772833304576,"internally_reserved":0,"allocated":5283840,"data_stored":2989824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.642910"},"pg_stats":[{"pgid":"1.0","version":"43'139","reported_seq":290,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T23:37:39.937258+0000","last_change":"2024-09-15T23:36:11.881330+0000","last_active":"2024-09-15T23:37:39.937258+0000","last_peered":"2024-09-15T23:37:39.937258+0000","last_clean":"2024-09-15T23:37:39.937258+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:39.937258+0000","last_undegraded":"2024-09-15T23:37:39.937258+0000","last_fullsized":"2024-09-15T23:37:39.937258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:36:09.861159+0000","last_clean_scrub_stamp":"2024-09-15T23:36:09.861159+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:30.396210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31696,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1105920,"data_stored":816504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31700,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1105920,"data_stored":816504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:37:48.730 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T23:36:09.861159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=23, tm_min=36, tm_sec=9, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=23, tm_min=37, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T23:37:48.730 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T23:37:55.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:37:55.785 DEBUG:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:38:08.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T23:38:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.847+0000 7fca177e8640 1 Processor -- start 2024-09-15T23:38:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.848+0000 7fca177e8640 1 -- start start 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.848+0000 7fca177e8640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca1015e840 0x7fca100d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.848+0000 7fca177e8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 0x7fca1006d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.848+0000 7fca177e8640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca100cf560 con 0x7fca1006dbb0 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.848+0000 7fca177e8640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fca100cfe90 con 0x7fca1015e840 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.848+0000 7fca177e8640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fca100cf840 con 0x7fca100d52e0 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca15584640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca1015e840 0x7fca100d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca15584640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca1015e840 0x7fca100d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46572/0 (socket says 172.21.15.7:46572) 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca15584640 1 -- 172.21.15.7:0/899206457 learned_addr learned my addr 172.21.15.7:0/899206457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca15d85640 1 --1- 172.21.15.7:0/899206457 >> v1:172.21.15.7:6789/0 conn(0x7fca1006dbb0 0x7fca100d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54490/0 (socket says 172.21.15.7:54490) 2024-09-15T23:38:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca14d83640 1 --2- 172.21.15.7:0/899206457 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 0x7fca1006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca15584640 1 -- 172.21.15.7:0/899206457 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 msgr2=0x7fca1006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:38:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca15584640 1 --2- 172.21.15.7:0/899206457 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 0x7fca1006d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca15584640 1 -- 172.21.15.7:0/899206457 >> v1:172.21.15.7:6789/0 conn(0x7fca1006dbb0 legacy=0x7fca100d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca15584640 1 -- 172.21.15.7:0/899206457 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca100cfb20 con 0x7fca1015e840 2024-09-15T23:38:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.849+0000 7fca07fff640 1 -- 172.21.15.7:0/899206457 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629151142 0 0) 0x7fca100cf560 con 0x7fca1006dbb0 2024-09-15T23:38:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.850+0000 7fca15584640 1 --2- 172.21.15.7:0/899206457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 0x7fca100d0320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009870 tx=0x7fca0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24974501fc459c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:38:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.850+0000 7fca07fff640 1 -- 172.21.15.7:0/899206457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037070 con 0x7fca1015e840 2024-09-15T23:38:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.850+0000 7fca07fff640 1 -- 172.21.15.7:0/899206457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c032440 con 0x7fca1015e840 2024-09-15T23:38:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.850+0000 7fca07fff640 1 -- 172.21.15.7:0/899206457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c032810 con 0x7fca1015e840 2024-09-15T23:38:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.851+0000 7fca177e8640 1 -- 172.21.15.7:0/899206457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 msgr2=0x7fca100d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.851+0000 7fca177e8640 1 --2- 172.21.15.7:0/899206457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 0x7fca100d0320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009870 tx=0x7fca0c02a270 comp rx=0 tx=0).stop 2024-09-15T23:38:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.851+0000 7fca177e8640 1 -- 172.21.15.7:0/899206457 shutdown_connections 2024-09-15T23:38:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.851+0000 7fca177e8640 1 --2- 172.21.15.7:0/899206457 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 0x7fca1006d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.851+0000 7fca177e8640 1 --2- 172.21.15.7:0/899206457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 0x7fca100d0320 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:08.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.851+0000 7fca177e8640 1 -- 172.21.15.7:0/899206457 >> 172.21.15.7:0/899206457 conn(0x7fca1008bfd0 msgr2=0x7fca100d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:38:08.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.851+0000 7fca177e8640 1 -- 172.21.15.7:0/899206457 shutdown_connections 2024-09-15T23:38:08.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.851+0000 7fca177e8640 1 -- 172.21.15.7:0/899206457 wait complete. 2024-09-15T23:38:08.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.852+0000 7fca177e8640 1 Processor -- start 2024-09-15T23:38:08.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.852+0000 7fca177e8640 1 -- start start 2024-09-15T23:38:08.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca177e8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca1006dbb0 0x7fca100cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca177e8640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 0x7fca100b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca177e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 0x7fca1014f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca177e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca1015ec20 con 0x7fca1015e840 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca177e8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fca100d0860 con 0x7fca1006dbb0 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca177e8640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7fca100b61c0 con 0x7fca100d52e0 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15d85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 0x7fca1014f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15d85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 0x7fca1014f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46574/0 (socket says 172.21.15.7:46574) 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15d85640 1 -- 172.21.15.7:0/2469882468 learned_addr learned my addr 172.21.15.7:0/2469882468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca14d83640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 0x7fca100b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15584640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca1006dbb0 0x7fca100cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15d85640 1 -- 172.21.15.7:0/2469882468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 msgr2=0x7fca100b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15d85640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 0x7fca100b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:08.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15d85640 1 -- 172.21.15.7:0/2469882468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca1006dbb0 msgr2=0x7fca100cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15d85640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca1006dbb0 0x7fca100cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15d85640 1 -- 172.21.15.7:0/2469882468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca100b55b0 con 0x7fca1015e840 2024-09-15T23:38:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.853+0000 7fca15d85640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 0x7fca1014f220 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fca0800c780 tx=0x7fca0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:38:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.854+0000 7fca05ffb640 1 -- 172.21.15.7:0/2469882468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08018070 con 0x7fca1015e840 2024-09-15T23:38:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.854+0000 7fca05ffb640 1 -- 172.21.15.7:0/2469882468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0800ce30 con 0x7fca1015e840 2024-09-15T23:38:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.854+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca100b5890 con 0x7fca1015e840 2024-09-15T23:38:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.854+0000 7fca05ffb640 1 -- 172.21.15.7:0/2469882468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0800a840 con 0x7fca1015e840 2024-09-15T23:38:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.854+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca100b5e50 con 0x7fca1015e840 2024-09-15T23:38:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.855+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1006d670 con 0x7fca1015e840 2024-09-15T23:38:08.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.856+0000 7fca05ffb640 1 -- 172.21.15.7:0/2469882468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7fca0801b070 con 0x7fca1015e840 2024-09-15T23:38:08.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.857+0000 7fca05ffb640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fc9f406c2e0 0x7fc9f406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:08.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.857+0000 7fca05ffb640 1 -- 172.21.15.7:0/2469882468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fca0808a9e0 con 0x7fca1015e840 2024-09-15T23:38:08.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.857+0000 7fca15584640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fc9f406c2e0 0x7fc9f406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:08.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.857+0000 7fca15584640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fc9f406c2e0 0x7fc9f406e7a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca0c034000 tx=0x7fca0c02e000 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:38:08.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:08.861+0000 7fca05ffb640 1 -- 172.21.15.7:0/2469882468 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca08058ad0 con 0x7fca1015e840 2024-09-15T23:38:09.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.035+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 --> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca100d1cb0 con 0x7fc9f406c2e0 2024-09-15T23:38:09.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.036+0000 7fca05ffb640 1 -- 172.21.15.7:0/2469882468 <== mgr.5172 v2:172.21.15.7:6835/1497838406 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13358 (secure 0 0 0) 0x7fca100d1cb0 con 0x7fc9f406c2e0 2024-09-15T23:38:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:38:09.039 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-15T23:38:09.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fc9f406c2e0 msgr2=0x7fc9f406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fc9f406c2e0 0x7fc9f406e7a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca0c034000 tx=0x7fca0c02e000 comp rx=0 tx=0).stop 2024-09-15T23:38:09.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 msgr2=0x7fca1014f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 0x7fca1014f220 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fca0800c780 tx=0x7fca0800cc40 comp rx=0 tx=0).stop 2024-09-15T23:38:09.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 shutdown_connections 2024-09-15T23:38:09.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7fc9f406c2e0 0x7fc9f406e7a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1015e840 0x7fca1014f220 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fca100d52e0 0x7fca100b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 --2- 172.21.15.7:0/2469882468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca1006dbb0 0x7fca100cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 >> 172.21.15.7:0/2469882468 conn(0x7fca1008bfd0 msgr2=0x7fca100b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:38:09.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 shutdown_connections 2024-09-15T23:38:09.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.038+0000 7fca177e8640 1 -- 172.21.15.7:0/2469882468 wait complete. 2024-09-15T23:38:09.051 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-15T23:38:07.334463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":254676,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720044,"statfs":{"total":773094113280,"available":772833325056,"internally_reserved":0,"allocated":5283840,"data_stored":2989824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002733"},"pg_stats":[{"pgid":"1.0","version":"44'140","reported_seq":300,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T23:37:46.635169+0000","last_change":"2024-09-15T23:37:46.635229+0000","last_active":"2024-09-15T23:37:46.635169+0000","last_peered":"2024-09-15T23:37:46.635169+0000","last_clean":"2024-09-15T23:37:46.635169+0000","last_became_active":"2024-09-15T23:36:11.880357+0000","last_became_peered":"2024-09-15T23:36:11.880357+0000","last_unstale":"2024-09-15T23:37:46.635169+0000","last_undegraded":"2024-09-15T23:37:46.635169+0000","last_fullsized":"2024-09-15T23:37:46.635169+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'140","last_scrub_stamp":"2024-09-15T23:37:46.635087+0000","last_deep_scrub":"44'140","last_deep_scrub_stamp":"2024-09-15T23:37:46.635087+0000","last_clean_scrub_stamp":"2024-09-15T23:37:46.635087+0000","objects_scrubbed":2,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:56.004364+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31696,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1105920,"data_stored":816504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31700,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1105920,"data_stored":816504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":226136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:38:09.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T23:38:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.155+0000 7f7ddc9d6640 1 Processor -- start 2024-09-15T23:38:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.155+0000 7f7ddc9d6640 1 -- start start 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.156+0000 7f7ddc9d6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 0x7f7dd80d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.156+0000 7f7ddc9d6640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd8163170 con 0x7f7dd815e5b0 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.156+0000 7f7ddc9d6640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd80d2980 con 0x7f7dd80d24e0 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.156+0000 7f7ddc9d6640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd80d2c40 con 0x7f7dd815e990 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.156+0000 7f7dd659c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7dd815e5b0 0x7f7dd8162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54492/0 (socket says 172.21.15.7:54492) 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.156+0000 7f7dd659c640 1 -- 172.21.15.7:0/936411230 learned_addr learned my addr 172.21.15.7:0/936411230 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.157+0000 7f7dd6d9d640 1 --2- 172.21.15.7:0/936411230 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 0x7f7dd80d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.157+0000 7f7dd559a640 1 -- 172.21.15.7:0/936411230 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3346314476 0 0) 0x7f7dd8163170 con 0x7f7dd815e5b0 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.157+0000 7f7dd559a640 1 -- 172.21.15.7:0/936411230 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dc0003520 con 0x7f7dd815e5b0 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.157+0000 7f7dd559a640 1 -- 172.21.15.7:0/936411230 <== mon.1 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845697268 0 0) 0x7f7dd80d2980 con 0x7f7dd80d24e0 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.157+0000 7f7dd559a640 1 -- 172.21.15.7:0/936411230 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dd8163170 con 0x7f7dd80d24e0 2024-09-15T23:38:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.157+0000 7f7dd559a640 1 -- 172.21.15.7:0/936411230 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1139091196 0 0) 0x7f7dc0003520 con 0x7f7dd815e5b0 2024-09-15T23:38:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.157+0000 7f7dd6d9d640 1 -- 172.21.15.7:0/936411230 >> v1:172.21.15.18:6789/0 conn(0x7f7dd80d24e0 legacy=0x7f7dd8162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.157+0000 7f7dd6d9d640 1 -- 172.21.15.7:0/936411230 >> v1:172.21.15.7:6789/0 conn(0x7f7dd815e5b0 legacy=0x7f7dd8162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.157+0000 7f7dd6d9d640 1 -- 172.21.15.7:0/936411230 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd80d2f20 con 0x7f7dd815e990 2024-09-15T23:38:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.158+0000 7f7dd6d9d640 1 --2- 172.21.15.7:0/936411230 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 0x7f7dd80d0430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009df0 tx=0x7f7dd002a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa1fbb406a00ddd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:38:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.158+0000 7f7dd559a640 1 -- 172.21.15.7:0/936411230 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80015f0 con 0x7f7dd815e990 2024-09-15T23:38:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.158+0000 7f7dd559a640 1 -- 172.21.15.7:0/936411230 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd0007540 con 0x7f7dd815e990 2024-09-15T23:38:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.158+0000 7f7dd559a640 1 -- 172.21.15.7:0/936411230 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0007710 con 0x7f7dd815e990 2024-09-15T23:38:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.159+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/936411230 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 msgr2=0x7f7dd80d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.159+0000 7f7ddc9d6640 1 --2- 172.21.15.7:0/936411230 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 0x7f7dd80d0430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009df0 tx=0x7f7dd002a6a0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.159+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/936411230 shutdown_connections 2024-09-15T23:38:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.159+0000 7f7ddc9d6640 1 --2- 172.21.15.7:0/936411230 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 0x7f7dd80d0430 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.159+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/936411230 >> 172.21.15.7:0/936411230 conn(0x7f7dd808bfd0 msgr2=0x7f7dd80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:38:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.159+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/936411230 shutdown_connections 2024-09-15T23:38:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.159+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/936411230 wait complete. 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.160+0000 7f7ddc9d6640 1 Processor -- start 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.160+0000 7f7ddc9d6640 1 -- start start 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7ddc9d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd80d24e0 0x7f7dd80cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7ddc9d6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7dd815e5b0 0x7f7dd80b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7ddc9d6640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 0x7f7dd814db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7ddc9d6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd80b39c0 con 0x7f7dd80d24e0 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7ddc9d6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd80d34e0 con 0x7f7dd815e5b0 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7ddc9d6640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd80b3b40 con 0x7f7dd815e990 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd80d24e0 0x7f7dd80cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd80d24e0 0x7f7dd80cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46584/0 (socket says 172.21.15.7:46584) 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd659c640 1 -- 172.21.15.7:0/1802895893 learned_addr learned my addr 172.21.15.7:0/1802895893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd6d9d640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 0x7f7dd814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd5d9b640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7dd815e5b0 0x7f7dd80b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd659c640 1 -- 172.21.15.7:0/1802895893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 msgr2=0x7f7dd814db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd659c640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 0x7f7dd814db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd659c640 1 -- 172.21.15.7:0/1802895893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7dd815e5b0 msgr2=0x7f7dd80b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd659c640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7dd815e5b0 0x7f7dd80b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd659c640 1 -- 172.21.15.7:0/1802895893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd80cd390 con 0x7f7dd80d24e0 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.161+0000 7f7dd5d9b640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7dd815e5b0 0x7f7dd80b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.162+0000 7f7dd659c640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd80d24e0 0x7f7dd80cccd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80011e0 tx=0x7f7dc800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.162+0000 7f7dc6ffd640 1 -- 172.21.15.7:0/1802895893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8017070 con 0x7f7dd80d24e0 2024-09-15T23:38:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.162+0000 7f7dc6ffd640 1 -- 172.21.15.7:0/1802895893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc8005ad0 con 0x7f7dd80d24e0 2024-09-15T23:38:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.162+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd80cd670 con 0x7f7dd80d24e0 2024-09-15T23:38:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.162+0000 7f7dc6ffd640 1 -- 172.21.15.7:0/1802895893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8007070 con 0x7f7dd80d24e0 2024-09-15T23:38:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.162+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd80cdc30 con 0x7f7dd80d24e0 2024-09-15T23:38:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.163+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd80ba560 con 0x7f7dd80d24e0 2024-09-15T23:38:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.164+0000 7f7dc6ffd640 1 -- 172.21.15.7:0/1802895893 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f7dc801a070 con 0x7f7dd80d24e0 2024-09-15T23:38:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.164+0000 7f7dc6ffd640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7db806c2b0 0x7f7db806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.164+0000 7f7dc6ffd640 1 -- 172.21.15.7:0/1802895893 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f7dc808a8c0 con 0x7f7dd80d24e0 2024-09-15T23:38:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.164+0000 7f7dd5d9b640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7db806c2b0 0x7f7db806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.165+0000 7f7dd5d9b640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7db806c2b0 0x7f7db806e770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7dd806ea00 tx=0x7f7dcc009040 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:38:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.169+0000 7f7dc6ffd640 1 -- 172.21.15.7:0/1802895893 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7dc8058980 con 0x7f7dd80d24e0 2024-09-15T23:38:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.347+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f7dd815d080 con 0x7f7dd80d24e0 2024-09-15T23:38:09.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.350+0000 7f7dc6ffd640 1 -- 172.21.15.7:0/1802895893 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7f7dc805cc80 con 0x7f7dd80d24e0 2024-09-15T23:38:09.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.352+0000 7f7dc6ffd640 1 -- 172.21.15.7:0/1802895893 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7dc80614b0 con 0x7f7dd80d24e0 2024-09-15T23:38:09.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7db806c2b0 msgr2=0x7f7db806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7db806c2b0 0x7f7db806e770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7dd806ea00 tx=0x7f7dcc009040 comp rx=0 tx=0).stop 2024-09-15T23:38:09.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd80d24e0 msgr2=0x7f7dd80cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd80d24e0 0x7f7dd80cccd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80011e0 tx=0x7f7dc800cbc0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 shutdown_connections 2024-09-15T23:38:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f7db806c2b0 0x7f7db806e770 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7dd815e990 0x7f7dd814db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7dd815e5b0 0x7f7dd80b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 --2- 172.21.15.7:0/1802895893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd80d24e0 0x7f7dd80cccd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 >> 172.21.15.7:0/1802895893 conn(0x7f7dd808bfd0 msgr2=0x7f7dd815d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:38:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 shutdown_connections 2024-09-15T23:38:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.353+0000 7f7ddc9d6640 1 -- 172.21.15.7:0/1802895893 wait complete. 2024-09-15T23:38:09.366 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-15T23:38:09.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T23:38:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.469+0000 7f8bd8482640 1 Processor -- start 2024-09-15T23:38:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.469+0000 7f8bd8482640 1 -- start start 2024-09-15T23:38:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.470+0000 7f8bd8482640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bd015fd10 0x7f8bd00b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.470+0000 7f8bd8482640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bd009bcf0 con 0x7f8bd009b150 2024-09-15T23:38:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.470+0000 7f8bd8482640 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bd00d09f0 con 0x7f8bd009b780 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.470+0000 7f8bd8482640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd00d0cd0 con 0x7f8bd015fd10 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.470+0000 7f8bd6a1f640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8bd009b150 0x7f8bd00cd2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54496/0 (socket says 172.21.15.7:54496) 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.470+0000 7f8bd6a1f640 1 -- 172.21.15.7:0/2193370466 learned_addr learned my addr 172.21.15.7:0/2193370466 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd621e640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bd015fd10 0x7f8bd00b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1617419504 0 0) 0x7f8bd009bcf0 con 0x7f8bd009b150 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bbc003520 con 0x7f8bd009b150 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4062832618 0 0) 0x7f8bbc003520 con 0x7f8bd009b150 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bd015fd10 msgr2=0x7f8bd00b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bd015fd10 0x7f8bd00b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 >> v1:172.21.15.18:6789/0 conn(0x7f8bd009b780 legacy=0x7f8bd009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd00d0fb0 con 0x7f8bd009b150 2024-09-15T23:38:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2020534986 0 0) 0x7f8bc80019f0 con 0x7f8bd009b150 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 >> v1:172.21.15.7:6789/0 conn(0x7f8bd009b150 legacy=0x7f8bd00cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bbc0039e0 0x7f8bbc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bd015fd10 0x7f8bbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bbc004d40 0x7f8bbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00d0cd0 con 0x7f8bbc0039e0 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc00b9c0 con 0x7f8bbc004d40 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8bbc00bbc0 con 0x7f8bd015fd10 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd621e640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bbc0039e0 0x7f8bbc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd5a1d640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bd015fd10 0x7f8bbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd6a1f640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bbc004d40 0x7f8bbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd621e640 1 -- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bd015fd10 msgr2=0x7f8bbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd621e640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bd015fd10 0x7f8bbc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd621e640 1 -- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bbc004d40 msgr2=0x7f8bbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:38:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.471+0000 7f8bd621e640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bbc004d40 0x7f8bbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.472+0000 7f8bd621e640 1 -- 172.21.15.7:0/2193370466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc003520 con 0x7f8bbc0039e0 2024-09-15T23:38:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.472+0000 7f8bd6a1f640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bbc004d40 0x7f8bbc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:38:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.472+0000 7f8bd621e640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bbc0039e0 0x7f8bbc003dc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009ac0 tx=0x7f8bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be67cc00276adc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:38:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.472+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4036070 con 0x7f8bbc0039e0 2024-09-15T23:38:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.472+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8bc4031400 con 0x7f8bbc0039e0 2024-09-15T23:38:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.472+0000 7f8bd521c640 1 -- 172.21.15.7:0/2193370466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4009690 con 0x7f8bbc0039e0 2024-09-15T23:38:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.473+0000 7f8bd8482640 1 -- 172.21.15.7:0/2193370466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bbc0039e0 msgr2=0x7f8bbc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.473+0000 7f8bd8482640 1 --2- 172.21.15.7:0/2193370466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bbc0039e0 0x7f8bbc003dc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009ac0 tx=0x7f8bbc005830 comp rx=0 tx=0).stop 2024-09-15T23:38:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.473+0000 7f8bd6a1f640 1 -- 172.21.15.7:0/2193370466 reap_dead start 2024-09-15T23:38:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.473+0000 7f8bd8482640 1 -- 172.21.15.7:0/2193370466 shutdown_connections 2024-09-15T23:38:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.473+0000 7f8bd8482640 1 -- 172.21.15.7:0/2193370466 >> 172.21.15.7:0/2193370466 conn(0x7f8bd00d2410 msgr2=0x7f8bd00d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:38:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.473+0000 7f8bd8482640 1 -- 172.21.15.7:0/2193370466 shutdown_connections 2024-09-15T23:38:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.473+0000 7f8bd8482640 1 -- 172.21.15.7:0/2193370466 wait complete. 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.474+0000 7f8bd8482640 1 Processor -- start 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.474+0000 7f8bd8482640 1 -- start start 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd8482640 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bbc0039e0 0x7f8bd01f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd8482640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bd015ce40 0x7f8bd00c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd8482640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd00b13f0 0x7f8bd00b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd8482640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00cecf0 con 0x7f8bd00b13f0 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd8482640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00ceb70 con 0x7f8bd015ce40 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd8482640 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd00b30d0 con 0x7f8bbc0039e0 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd6a1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd00b13f0 0x7f8bd00b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd6a1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd00b13f0 0x7f8bd00b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46604/0 (socket says 172.21.15.7:46604) 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd6a1f640 1 -- 172.21.15.7:0/2408119970 learned_addr learned my addr 172.21.15.7:0/2408119970 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd5a1d640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bd015ce40 0x7f8bd00c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.475+0000 7f8bd621e640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bbc0039e0 0x7f8bd01f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bd6a1f640 1 -- 172.21.15.7:0/2408119970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bbc0039e0 msgr2=0x7f8bd01f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bd6a1f640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bbc0039e0 0x7f8bd01f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bd6a1f640 1 -- 172.21.15.7:0/2408119970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bd015ce40 msgr2=0x7f8bd00c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bd6a1f640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bd015ce40 0x7f8bd00c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bd6a1f640 1 -- 172.21.15.7:0/2408119970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd01f1ed0 con 0x7f8bd00b13f0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bd6a1f640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd00b13f0 0x7f8bd00b18a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0073ec0 tx=0x7f8bc800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bc2ffd640 1 -- 172.21.15.7:0/2408119970 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800eda0 con 0x7f8bd00b13f0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bc2ffd640 1 -- 172.21.15.7:0/2408119970 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8bc8018070 con 0x7f8bd00b13f0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd00c60e0 con 0x7f8bd00b13f0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bc2ffd640 1 -- 172.21.15.7:0/2408119970 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8013660 con 0x7f8bd00b13f0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.476+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd00c66a0 con 0x7f8bd00b13f0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.477+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd009aa40 con 0x7f8bd00b13f0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.478+0000 7f8bc2ffd640 1 -- 172.21.15.7:0/2408119970 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 90846+0+0 (secure 0 0 0) 0x7f8bc8013800 con 0x7f8bd00b13f0 2024-09-15T23:38:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.478+0000 7f8bc2ffd640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f8bb406c2b0 0x7f8bb406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:38:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.478+0000 7f8bd621e640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f8bb406c2b0 0x7f8bb406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:38:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.478+0000 7f8bc2ffd640 1 -- 172.21.15.7:0/2408119970 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f8bc808b890 con 0x7f8bd00b13f0 2024-09-15T23:38:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.479+0000 7f8bd621e640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f8bb406c2b0 0x7f8bb406e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4033000 tx=0x7f8bc4008030 comp rx=0 tx=0).ready entity=mgr.5172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:38:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.483+0000 7f8bc2ffd640 1 -- 172.21.15.7:0/2408119970 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8bc8059830 con 0x7f8bd00b13f0 2024-09-15T23:38:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.696+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd00d3000 con 0x7f8bd00b13f0 2024-09-15T23:38:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.696+0000 7f8bc2ffd640 1 -- 172.21.15.7:0/2408119970 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+3902 (secure 0 0 0) 0x7f8bc805db30 con 0x7f8bd00b13f0 2024-09-15T23:38:09.698 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-15T23:38:09.698 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"btime":"2024-09-15T23:37:37:679267+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1101755994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6833","nonce":1101755994},{"type":"v1","addr":"172.21.15.18:6835","nonce":1101755994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6834/2816400413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2816400413},{"type":"v1","addr":"172.21.15.18:6834","nonce":2816400413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4827,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/809864756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":809864756},{"type":"v1","addr":"172.21.15.7:6836","nonce":809864756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3076953893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3076953893},{"type":"v1","addr":"172.21.15.7:6833","nonce":3076953893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[]} 2024-09-15T23:38:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-09-15T23:38:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f8bb406c2b0 msgr2=0x7f8bb406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f8bb406c2b0 0x7f8bb406e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4033000 tx=0x7f8bc4008030 comp rx=0 tx=0).stop 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd00b13f0 msgr2=0x7f8bd00b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd00b13f0 0x7f8bd00b18a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0073ec0 tx=0x7f8bc800e530 comp rx=0 tx=0).stop 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 shutdown_connections 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:6835/1497838406,v1:172.21.15.7:6837/1497838406] conn(0x7f8bb406c2b0 0x7f8bb406e770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd00b13f0 0x7f8bd00b18a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bd015ce40 0x7f8bd00c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 --2- 172.21.15.7:0/2408119970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8bbc0039e0 0x7f8bd01f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 >> 172.21.15.7:0/2408119970 conn(0x7f8bd00d2410 msgr2=0x7f8bd00cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:38:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 shutdown_connections 2024-09-15T23:38:09.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-15T23:38:09.698+0000 7f8bd8482640 1 -- 172.21.15.7:0/2408119970 wait complete. 2024-09-15T23:38:09.711 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-15T23:38:09.711 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T23:38:09.711 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-15T23:38:09.712 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:09.722 INFO:tasks.ceph.mds.a:Stopped 2024-09-15T23:38:09.723 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-15T23:38:09.723 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:09.772 INFO:tasks.ceph.mds.c:Stopped 2024-09-15T23:38:09.772 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-15T23:38:09.773 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:09.785 INFO:tasks.ceph.mds.b:Stopped 2024-09-15T23:38:09.785 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-15T23:38:09.785 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:09.837 INFO:tasks.ceph.mds.d:Stopped 2024-09-15T23:38:09.837 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T23:38:09.838 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T23:38:09.838 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:09.868 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T23:38:09.869 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T23:38:09.869 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:09.939 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T23:38:09.939 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T23:38:09.939 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.013 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T23:38:10.013 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T23:38:10.013 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.080 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T23:38:10.081 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T23:38:10.081 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.114 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T23:38:10.114 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T23:38:10.114 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.182 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T23:38:10.182 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-15T23:38:10.182 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.257 INFO:tasks.ceph.osd.6:Stopped 2024-09-15T23:38:10.258 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-15T23:38:10.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.334 INFO:tasks.ceph.osd.7:Stopped 2024-09-15T23:38:10.334 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T23:38:10.335 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-15T23:38:10.335 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.367 INFO:tasks.ceph.mgr.y:Stopped 2024-09-15T23:38:10.367 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T23:38:10.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.405 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T23:38:10.405 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T23:38:10.406 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T23:38:10.406 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.419 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T23:38:10.419 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T23:38:10.419 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.458 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T23:38:10.458 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T23:38:10.458 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T23:38:10.512 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T23:38:10.513 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T23:38:10.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-15T23:38:10.560 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:38:10.560 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T23:38:10.723 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:38:10.723 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T23:38:10.814 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:38:10.815 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T23:38:10.904 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:38:10.905 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T23:38:10.954 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:38:10.955 DEBUG:teuthology.orchestra.run.smithi018:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T23:38:11.097 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:38:11.097 DEBUG:teuthology.orchestra.run.smithi018:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T23:38:11.176 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:38:11.177 DEBUG:teuthology.orchestra.run.smithi018:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-15T23:38:11.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:38:11.235 DEBUG:teuthology.orchestra.run.smithi018:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-15T23:38:11.284 INFO:tasks.ceph:Archiving mon data... 2024-09-15T23:38:11.285 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331/data/mon.a.tgz 2024-09-15T23:38:11.286 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-15T23:38:11.302 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.zKshjFAo1e 2024-09-15T23:38:11.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.zKshjFAo1e 2024-09-15T23:38:11.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.zKshjFAo1e 2024-09-15T23:38:11.596 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.zKshjFAo1e is 712KB 2024-09-15T23:38:11.695 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.zKshjFAo1e 2024-09-15T23:38:11.711 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331/data/mon.b.tgz 2024-09-15T23:38:11.711 DEBUG:teuthology.orchestra.run.smithi018:> mktemp 2024-09-15T23:38:11.730 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.Wx6HBei7kP 2024-09-15T23:38:11.731 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Wx6HBei7kP 2024-09-15T23:38:11.976 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.Wx6HBei7kP 2024-09-15T23:38:12.061 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.Wx6HBei7kP is 1020KB 2024-09-15T23:38:12.166 DEBUG:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.Wx6HBei7kP 2024-09-15T23:38:12.182 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331/data/mon.c.tgz 2024-09-15T23:38:12.182 DEBUG:teuthology.orchestra.run.smithi018:> mktemp 2024-09-15T23:38:12.237 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.uhIkeJqp12 2024-09-15T23:38:12.237 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.uhIkeJqp12 2024-09-15T23:38:12.469 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.uhIkeJqp12 2024-09-15T23:38:12.554 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.uhIkeJqp12 is 1020KB 2024-09-15T23:38:12.632 DEBUG:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.uhIkeJqp12 2024-09-15T23:38:12.647 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T23:38:12.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T23:38:12.651 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T23:38:12.737 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T23:38:12.769 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T23:38:12.769 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T23:38:12.771 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331/remote/smithi007/crash 2024-09-15T23:38:12.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T23:38:12.806 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331/remote/smithi018/crash 2024-09-15T23:38:12.807 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T23:38:12.833 INFO:tasks.ceph:Shutting down logrotate 2024-09-15T23:38:12.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:38:12.851 DEBUG:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T23:38:12.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-15T23:38:12.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-15T23:38:12.975 INFO:tasks.ceph:Compressing logs... 2024-09-15T23:38:12.975 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T23:38:12.992 DEBUG:teuthology.orchestra.run.smithi018:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T23:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T23:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43630.log 2024-09-15T23:38:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42051.log 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43672.log 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42051.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.43630.log.gz -5 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47989.log 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42051.log.gz 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45786.log 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43672.log.gz 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45611.log 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47989.log.gz 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42093.log 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.2.45611.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45786.log.gz 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43854.log 2024-09-15T23:38:13.049 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45611.log.gz 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46098.log 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42093.log.gz 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51154.log 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46718.log 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43854.log.gz 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51154.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46098.log.gz 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50919.log 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51154.log.gz 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43175.log 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50919.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46718.log.gz 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46676.log 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50919.log.gz 2024-09-15T23:38:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50415.log 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43175.log.gz 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47470.log 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50415.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46676.log.gz 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48429.log 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50415.log.gz 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42239.log 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47470.log.gz 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.48429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43504.log 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48429.log.gz 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44188.log 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42239.log.gz 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46298.log 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43504.log.gz 2024-09-15T23:38:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45170.log 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46298.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44188.log.gz 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46298.log.gz 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.45170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49500.log 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45170.log.gz 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50373.log 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2%/var/log/ceph/ceph-client.admin.50373.log: -- replaced with /var/log/ceph/ceph-client.admin.49500.log.gz 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46550.log 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50373.log.gz 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47649.log 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44596.log 2024-09-15T23:38:13.052 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46550.log.gz 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47695.log 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47649.log.gz 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46844.log 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49972.log 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44596.log.gz 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47342.log 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47695.log.gz 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46844.log.gz 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50835.log 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49972.log.gz 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48553.log 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47342.log.gz 2024-09-15T23:38:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50835.log: /var/log/ceph/ceph-client.admin.48553.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50835.log.gz 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44187.log 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43853.log 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48553.log.gz 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46056.log: /var/log/ceph/ceph-client.admin.44187.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42197.log 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47565.log 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44187.log.gz 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43853.log: /var/log/ceph/ceph-client.admin.42197.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43853.log.gz 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51003.log 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46928.log 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42197.log.gz 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47565.log: /var/log/ceph/ceph-client.admin.51003.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47565.log.gz 2024-09-15T23:38:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46210.log 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49066.log 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51003.log.gz 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46928.log: /var/log/ceph/ceph-client.admin.46210.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46928.log.gz 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45744.log 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46210.log.gz 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46014.log 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49066.log: /var/log/ceph/ceph-client.admin.45744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41755.log 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49066.log.gz 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46014.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45744.log.gz 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46466.log 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46014.log.gz 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48894.log 2024-09-15T23:38:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41755.log: /var/log/ceph/ceph-client.admin.46466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42407.log 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41755.log.gz 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46466.log.gz 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45888.log 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48603.log 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42407.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48894.log.gz 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45888.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42407.log.gz 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50163.log 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45888.log.gz 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43846.log 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48603.log: /var/log/ceph/ceph-client.admin.50163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49416.log 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48603.log.gz 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50163.log.gz/var/log/ceph/ceph-client.admin.43846.log: 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47173.log 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43846.log.gz 2024-09-15T23:38:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50751.log 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49416.log.gz 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48031.log 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47173.log: /var/log/ceph/ceph-client.admin.50751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48720.log 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47173.log.gz 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48031.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50751.log.gz 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44470.log 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48031.log.gz 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47216.log 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48720.log: /var/log/ceph/ceph-client.admin.44470.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48720.log.gz 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50961.log 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44470.log.gz 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.47216.log: -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T23:38:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49374.log 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47216.log.gz 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50961.log.gz 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50793.log 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43336.log 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49374.log: /var/log/ceph/ceph-client.admin.50793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50625.log 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49374.log.gz 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50793.log.gz 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46886.log 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50625.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43336.log.gz 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48115.log 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50625.log.gz 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43117.log 2024-09-15T23:38:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46886.log: /var/log/ceph/ceph-client.admin.48115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47089.log 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46886.log.gz 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48115.log.gz 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50205.log 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48157.log 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47089.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43117.log.gz 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50205.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47089.log.gz 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50709.log 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50205.log.gz 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45165.log 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48157.log: /var/log/ceph/ceph-client.admin.50709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42515.log 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.45165.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50709.log.gz 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43546.log 0.0% 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.0.45165.log.gz 2024-09-15T23:38:13.059 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48157.log.gz 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44178.log 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43546.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42515.log.gz 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41567.log 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50541.log 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5%/var/log/ceph/ceph-client.admin.44178.log: -- replaced with /var/log/ceph/ceph-client.admin.43546.log.gz 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44178.log.gz 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41567.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30384.log 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47514.log 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50541.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41567.log.gz 2024-09-15T23:38:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.30384.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50541.log.gz 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30384.log.gz 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.47514.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44428.log 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.a.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47514.log.gz 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45972.log 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42281.log 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44428.log: /var/log/ceph/ceph-client.admin.45972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50583.log 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44428.log.gz 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42281.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45972.log.gz 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47821.log 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42281.log.gz 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43851.log 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47737.log 2024-09-15T23:38:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47821.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50583.log.gz 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47863.log 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47821.log.gz 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43851.log.gz 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41659.log 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47863.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47737.log.gz 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:gzip 88.7%/var/log/ceph/ceph-client.admin.41659.log: -5 -- replaced with /var/log/ceph/ceph-client.admin.47863.log.gz --verbose 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.44181.log 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41659.log.gz 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44386.log 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.44181.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44845.log 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44386.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44181.log.gz 2024-09-15T23:38:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47131.log 2024-09-15T23:38:13.063 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44386.log.gz 2024-09-15T23:38:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50667.log 2024-09-15T23:38:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47131.log: gzip -5 --verbose 89.5% -- /var/log/ceph/ceph-client.admin.48937.log -- replaced with /var/log/ceph/ceph-client.admin.44845.log.gz 2024-09-15T23:38:13.063 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-15T23:38:13.083 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36555.log 2024-09-15T23:38:13.083 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36424.log 2024-09-15T23:38:13.083 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.admin.36555.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-15T23:38:13.083 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.admin.36424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36384.log 2024-09-15T23:38:13.083 INFO:teuthology.orchestra.run.smithi018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36555.log.gz 2024-09-15T23:38:13.083 INFO:teuthology.orchestra.run.smithi018.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36424.log.gz 2024-09-15T23:38:13.083 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-15T23:38:13.083 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.admin.36384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41801.log 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36384.log.gz 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36515.log 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.3.41801.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41801.log.gz 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.admin.36515.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36468.log 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36515.log.gz 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36304.log 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.admin.36468.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36468.log.gz 2024-09-15T23:38:13.084 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.admin.36304.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-mon.c.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36304.log.gz -5 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr: --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36264.log 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.admin.36264.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36264.log.gz 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36344.log 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-mgr.x.log: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41383.log 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.admin.36344.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36344.log.gz 2024-09-15T23:38:13.085 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T23:38:13.086 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.1.41383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41383.log.gz 2024-09-15T23:38:13.086 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41797.log 2024-09-15T23:38:13.086 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-mon.b.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T23:38:13.086 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41378.log 2024-09-15T23:38:13.086 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.3.41797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41797.log.gz 2024-09-15T23:38:13.086 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.1.41378.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41378.log.gz 2024-09-15T23:38:13.087 INFO:teuthology.orchestra.run.smithi018.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47131.log.gz 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43252.log 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48937.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50667.log.gz 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42449.log 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48937.log.gz 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51063.log 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46508.log 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42449.log: /var/log/ceph/ceph-client.admin.51063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43294.log 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49542.log 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51063.log.gz 2024-09-15T23:38:13.090 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42449.log.gz 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46508.log: /var/log/ceph/ceph-client.admin.43294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46592.log 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46508.log.gz 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43294.log.gz 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47426.log 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49542.log.gz 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47384.log 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46592.log: /var/log/ceph/ceph-client.admin.47426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44186.log 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47426.log.gz 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46592.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47384.log: 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44186.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47384.log.gz 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46802.log 2024-09-15T23:38:13.091 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44186.log.gz 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50247.log 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44638.log 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50247.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46802.log.gz 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46389.log 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44638.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50247.log.gz 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44638.log.gz 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49321.log 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46389.log.gz 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41965.log 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44512.log 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49321.log.gz 2024-09-15T23:38:13.092 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47300.log 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41965.log: /var/log/ceph/ceph-client.admin.44512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36427.log 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41965.log.gz 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44512.log.gz 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47300.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47300.log.gz 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49458.log 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mgr.y.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36427.log.gz 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46634.log 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48511.log 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49458.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49458.log.gz 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46634.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46760.log 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46634.log.gz 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42942.log 2024-09-15T23:38:13.093 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48511.log.gz 2024-09-15T23:38:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47008.log 2024-09-15T23:38:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42942.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46760.log.gz 2024-09-15T23:38:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T23:38:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47008.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47008.log.gz 2024-09-15T23:38:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45303.log 2024-09-15T23:38:13.094 INFO:teuthology.orchestra.run.smithi007.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.42942.log.gz 2024-09-15T23:38:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42155.log 2024-09-15T23:38:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45303.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45303.log.gz 2024-09-15T23:38:13.094 INFO:teuthology.orchestra.run.smithi007.stderr: 84.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45930.log 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44183.log 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42155.log.gz 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49888.log 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45930.log.gz 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44183.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44183.log.gz 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47607.log 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49846.log 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49888.log: /var/log/ceph/ceph-client.admin.47607.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49888.log.gz 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47607.log.gz 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45846.log 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51104.log 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49846.log.gz 2024-09-15T23:38:13.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47905.log 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45846.log.gz 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44189.log 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51104.log.gz 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49279.log 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47905.log: /var/log/ceph/ceph-client.admin.44189.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47905.log.gz 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44189.log.gz 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50877.log 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49930.log 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49279.log.gz 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41879.log 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50877.log.gz 2024-09-15T23:38:13.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43842.log 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41879.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49930.log.gz 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41879.log.gz 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47258.log 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43420.log 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43842.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43842.log.gz 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47258.log.gz 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36781.log 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43420.log.gz 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36781.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36781.log.gz 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43843.log 2024-09-15T23:38:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49602.log 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43848.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43843.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50499.log 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43843.log.gz 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47947.log 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50499.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49602.log.gz 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50289.log 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50499.log.gz 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43462.log 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47947.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47947.log.gz 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36691.log 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50289.log.gz 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41837.log 2024-09-15T23:38:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47779.log 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43462.log.gz 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36691.log.gz 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50331.log 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41837.log: /var/log/ceph/ceph-client.admin.47779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43378.log 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41837.log.gz 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47779.log.gz 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42323.log 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50331.log.gz 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42323.log: gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-client.2.45607.log 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43378.log.gz 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42323.log.gz 2024-09-15T23:38:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49185.log 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43850.log: /var/log/ceph/ceph-client.2.45607.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45607.log.gz 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42365.log 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48073.log 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49185.log.gz 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50457.log 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42365.log.gz 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48661.log 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48073.log: /var/log/ceph/ceph-client.admin.50457.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48073.log.gz 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49751.log 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50457.log.gz 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42009.log 2024-09-15T23:38:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49751.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48661.log.gz 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48223.log 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49751.log.gz 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48433.log 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42009.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42009.log.gz 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44190.log 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.48433.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48223.log.gz 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48433.log.gz 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49793.log 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44190.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44190.log.gz 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41921.log 2024-09-15T23:38:13.101 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49793.log.gz 2024-09-15T23:38:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.c.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-15T23:38:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48979.log 2024-09-15T23:38:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41921.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41921.log.gz 2024-09-15T23:38:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48979.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48979.log.gz 2024-09-15T23:38:13.112 INFO:teuthology.orchestra.run.smithi007.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-15T23:38:13.134 INFO:teuthology.orchestra.run.smithi018.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T23:38:13.172 INFO:teuthology.orchestra.run.smithi018.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T23:38:13.247 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T23:38:13.707 INFO:teuthology.orchestra.run.smithi018.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-15T23:38:13.741 INFO:teuthology.orchestra.run.smithi018.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T23:38:13.744 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T23:38:13.758 INFO:teuthology.orchestra.run.smithi007.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T23:38:13.764 INFO:teuthology.orchestra.run.smithi018.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T23:38:13.770 INFO:teuthology.orchestra.run.smithi007.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T23:38:13.773 INFO:teuthology.orchestra.run.smithi018.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-15T23:38:13.774 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T23:38:13.775 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-09-15T23:38:13.775 INFO:teuthology.orchestra.run.smithi018.stderr:real 0m0.738s 2024-09-15T23:38:13.775 INFO:teuthology.orchestra.run.smithi018.stderr:user 0m2.946s 2024-09-15T23:38:13.775 INFO:teuthology.orchestra.run.smithi018.stderr:sys 0m0.117s 2024-09-15T23:38:13.776 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-15T23:38:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m0.770s 2024-09-15T23:38:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m3.129s 2024-09-15T23:38:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.238s 2024-09-15T23:38:13.776 INFO:tasks.ceph:Archiving logs... 2024-09-15T23:38:13.776 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331/remote/smithi007/log 2024-09-15T23:38:13.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T23:38:14.428 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331/remote/smithi018/log 2024-09-15T23:38:14.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T23:38:14.871 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T23:38:14.880 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T23:38:14.880 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T23:38:14.883 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T23:38:14.965 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T23:38:14.974 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T23:38:14.974 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:38:14.977 DEBUG:teuthology.orchestra.run.smithi018:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:38:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:38:14.993 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:38:14.995 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:38:14.995 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-15T23:38:14.995 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +481us[ +485us] +/- 52ms 2024-09-15T23:38:14.995 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -1322us[-1317us] +/- 79ms 2024-09-15T23:38:14.995 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +672us[ +677us] +/- 89ms 2024-09-15T23:38:14.995 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:38:14.998 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:38:14.999 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-09-15T23:38:14.999 INFO:teuthology.orchestra.run.smithi018.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +479us[ +483us] +/- 52ms 2024-09-15T23:38:14.999 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -1317us[-1313us] +/- 79ms 2024-09-15T23:38:14.999 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +665us[ +670us] +/- 89ms 2024-09-15T23:38:14.999 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:38:14.999 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T23:38:15.008 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T23:38:15.008 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T23:38:15.034 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T23:38:15.108 DEBUG:teuthology.orchestra.run.smithi018:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T23:38:15.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:38:15.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:38:15.206 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:38:15.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:38:15.236 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T23:38:15.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T23:38:15.269 INFO:teuthology.task.internal:Duration was 503.061038 seconds 2024-09-15T23:38:15.270 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T23:38:15.308 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T23:38:15.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T23:38:15.311 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T23:38:15.350 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:38:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:38:15.709 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T23:38:15.709 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-09-15T23:38:15.709 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T23:38:15.772 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi018.front.sepia.ceph.com 2024-09-15T23:38:15.772 DEBUG:teuthology.orchestra.run.smithi018:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T23:38:15.799 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T23:38:15.800 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T23:38:15.814 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T23:38:15.868 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T23:38:15.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T23:38:15.882 DEBUG:teuthology.orchestra.run.smithi018:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T23:38:16.011 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T23:38:16.021 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T23:38:16.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T23:38:16.047 DEBUG:teuthology.orchestra.run.smithi018:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T23:38:16.074 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T23:38:16.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:38:16.090 DEBUG:teuthology.orchestra.run.smithi018:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:38:16.112 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-09-15T23:38:16.139 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = core 2024-09-15T23:38:16.153 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:38:16.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:38:16.181 DEBUG:teuthology.orchestra.run.smithi018:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:38:16.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:38:16.210 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T23:38:16.220 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T23:38:16.221 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331/remote/smithi007 2024-09-15T23:38:16.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T23:38:16.256 DEBUG:teuthology.misc:Transferring archived files from smithi018:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906331/remote/smithi018 2024-09-15T23:38:16.256 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T23:38:16.289 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T23:38:16.290 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T23:38:16.294 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T23:38:16.340 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T23:38:16.351 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T23:38:16.351 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T23:38:16.386 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T23:38:16.387 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T23:38:16.389 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T23:38:16.402 INFO:teuthology.orchestra.run.smithi007.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 23:38 /home/ubuntu/cephtest 2024-09-15T23:38:16.405 INFO:teuthology.orchestra.run.smithi018.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 23:38 /home/ubuntu/cephtest 2024-09-15T23:38:16.406 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T23:38:16.418 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T23:38:16.488 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/snap_schedule_snapdir} duration: 503.0610382556915 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T23:38:16.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T23:38:16.575 INFO:teuthology.run:pass